[jira] [Commented] (HADOOP-13669) KMS Server should log exceptions before throwing
[ https://issues.apache.org/jira/browse/HADOOP-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547877#comment-15547877 ] Xiao Chen commented on HADOOP-13669: I agree those errors usually indicate a problem. But my concern is that, assuming we have 1 problematic key which throws exceptions, we may end up seeing that key log a bunch of stacktraces, hence spamming the log and also impacting server performance. Making it debug can give us more control (e.g. choose to temporarily turn it on on 1 KMS instance etc.) > KMS Server should log exceptions before throwing > > > Key: HADOOP-13669 > URL: https://issues.apache.org/jira/browse/HADOOP-13669 > Project: Hadoop Common > Issue Type: Improvement > Components: kms >Reporter: Xiao Chen >Assignee: Suraj Acharya > Labels: supportability > Attachments: HADOOP-13369.patch > > > In some recent investigation, it turns out when KMS throws an exception (into > tomcat), it's not logged anywhere and we can only see the exception message > from client-side, but not the stacktrace. Logging the stacktrance would help > debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13614) Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
[
https://issues.apache.org/jira/browse/HADOOP-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547731#comment-15547731
]
Chris Nauroth commented on HADOOP-13614:
[~ste...@apache.org], thank you for the nice work here. It looks good to me.
I have no other specific feedback. I would like to take one more review pass
over it tomorrow morning (PDT) with a clearer head, but I think it's highly
likely that I will approve after getting a clean pre-commit run. (As mentioned
earlier, I think the GitHub pull request needs to be updated to match the
revision 005 file to trigger pre-commit.)
> Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
> ---
>
> Key: HADOOP-13614
> URL: https://issues.apache.org/jira/browse/HADOOP-13614
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13614-branch-2-001.patch,
> HADOOP-13614-branch-2-002.patch, HADOOP-13614-branch-2-002.patch,
> HADOOP-13614-branch-2-004.patch, HADOOP-13614-branch-2-005.patch, testrun.txt
>
>
> Some of the slow test cases contain tests that are now obsoleted by newer
> ones. For example, {{ITestS3ADeleteManyFiles}} has the test case
> {{testOpenCreate()}} which writes then reads files up 25 MB.
> Have a look at which of the s3a tests are taking time, review them to see if
> newer tests have superceded the slow ones; and cut them where appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10075) Update jetty dependency to version 9
[ https://issues.apache.org/jira/browse/HADOOP-10075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kanter updated HADOOP-10075: --- Attachment: HADOOP-10075.007.patch The 007 patch: - No longer uses the com.github.phuonghuynh plugin. I've written one in hadoop-maven-plugins that does something similar instead. > Update jetty dependency to version 9 > > > Key: HADOOP-10075 > URL: https://issues.apache.org/jira/browse/HADOOP-10075 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.2.0, 2.6.0 >Reporter: Robert Rati >Assignee: Robert Kanter > Attachments: HADOOP-10075-002-wip.patch, HADOOP-10075.003.patch, > HADOOP-10075.004.patch, HADOOP-10075.005.patch, HADOOP-10075.006.patch, > HADOOP-10075.007.patch, HADOOP-10075.patch > > > Jetty6 is no longer maintained. Update the dependency to jetty9. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13522) Permissions on fs stat added
[
https://issues.apache.org/jira/browse/HADOOP-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546912#comment-15546912
]
Hadoop QA commented on HADOOP-13522:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
9s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
50s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 54s{color} | {color:orange} root: The patch generated 10 new + 238 unchanged
- 1 fixed = 248 total (was 239) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 5s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 79m 22s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ipc.TestIPC |
| | hadoop.hdfs.TestErasureCodeBenchmarkThroughput |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HADOOP-13522 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831607/HADOOP-13522.006.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux 79f9577e68d9 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 44f48ee |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10663/artifact/patchprocess/diff-checkstyle-root.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10663/artifact/patchprocess/patch-unit-hado
[jira] [Updated] (HADOOP-12010) Inter-operable between Java RS erasure coder and native RS erasure coder
[
https://issues.apache.org/jira/browse/HADOOP-12010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-12010:
-
Labels: hdfs-ec-3.0-must-do (was: )
> Inter-operable between Java RS erasure coder and native RS erasure coder
>
>
> Key: HADOOP-12010
> URL: https://issues.apache.org/jira/browse/HADOOP-12010
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Kai Zheng
>Assignee: Kai Zheng
> Labels: hdfs-ec-3.0-must-do
>
> It's natural and desired to support inter-operable between the two
> implemented RS raw erasure coders, in other words, data encoded in one coder
> (like {{RSRawEncoder}}) can be decoded using the other coder (like
> {{NativeRSRawDecoder}}). Without this, raw erasure coder will not be
> transparent to HDFS data. As such support isn't trivial and involves
> challenge work because the two implementations use different encode/decode
> matrix generation algorithms, better to have this separately.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12010) Inter-operable between Java RS erasure coder and native RS erasure coder
[
https://issues.apache.org/jira/browse/HADOOP-12010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-12010:
-
Affects Version/s: 3.0.0-alpha1
> Inter-operable between Java RS erasure coder and native RS erasure coder
>
>
> Key: HADOOP-12010
> URL: https://issues.apache.org/jira/browse/HADOOP-12010
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Kai Zheng
>Assignee: Kai Zheng
> Labels: hdfs-ec-3.0-must-do
>
> It's natural and desired to support inter-operable between the two
> implemented RS raw erasure coders, in other words, data encoded in one coder
> (like {{RSRawEncoder}}) can be decoded using the other coder (like
> {{NativeRSRawDecoder}}). Without this, raw erasure coder will not be
> transparent to HDFS data. As such support isn't trivial and involves
> challenge work because the two implementations use different encode/decode
> matrix generation algorithms, better to have this separately.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13683) Update jackson from 1.9.13 to 2.x in hadoop-hdfs
[
https://issues.apache.org/jira/browse/HADOOP-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546819#comment-15546819
]
Hadoop QA commented on HADOOP-13683:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} hadoop-hdfs-project: The patch generated 0 new + 163
unchanged - 1 fixed = 163 total (was 164) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
1s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 84m
38s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
8s{color} | {color:green} hadoop-hdfs-nfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}121m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HADOOP-13683 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831606/HADOOP-13683.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux aa1955423125 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29
20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 44f48ee |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10662/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client
hadoop-hdfs-project/hadoop-hdf
[jira] [Updated] (HADOOP-13061) Refactor erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HADOOP-13061: --- Labels: hdfs-ec-3.0-must-do (was: ) > Refactor erasure coders > --- > > Key: HADOOP-13061 > URL: https://issues.apache.org/jira/browse/HADOOP-13061 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Rui Li >Assignee: Kai Sasaki > Labels: hdfs-ec-3.0-must-do > Attachments: HADOOP-13061.01.patch, HADOOP-13061.02.patch, > HADOOP-13061.03.patch, HADOOP-13061.04.patch, HADOOP-13061.05.patch, > HADOOP-13061.06.patch, HADOOP-13061.07.patch, HADOOP-13061.08.patch, > HADOOP-13061.09.patch, HADOOP-13061.10.patch, HADOOP-13061.11.patch, > HADOOP-13061.12.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13665) Erasure Coding codec should support fallback coder
[
https://issues.apache.org/jira/browse/HADOOP-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Zhang updated HADOOP-13665:
---
Labels: hdfs-ec-3.0-must-do (was: )
> Erasure Coding codec should support fallback coder
> --
>
> Key: HADOOP-13665
> URL: https://issues.apache.org/jira/browse/HADOOP-13665
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: io
>Reporter: Wei-Chiu Chuang
> Labels: hdfs-ec-3.0-must-do
>
> The current EC codec supports a single coder only (by default pure Java
> implementation). If the native coder is specified but is unavailable, it
> should fallback to pure Java implementation.
> One possible solution is to follow the convention of existing Hadoop native
> codec, such as transport encryption (see {{CryptoCodec.java}}). It supports
> fallback by specifying two or multiple coders as the value of property, and
> loads coders in order.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13200) Seeking a better approach allowing to customize and configure erasure coders
[ https://issues.apache.org/jira/browse/HADOOP-13200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HADOOP-13200: --- Labels: hdfs-ec-3.0-must-do (was: ) > Seeking a better approach allowing to customize and configure erasure coders > > > Key: HADOOP-13200 > URL: https://issues.apache.org/jira/browse/HADOOP-13200 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Kai Zheng >Assignee: Kai Zheng > Labels: hdfs-ec-3.0-must-do > > This is a follow-on task for HADOOP-13010 as discussed over there. There may > be some better approach allowing to customize and configure erasure coders > than the current having raw coder factory, as [~cmccabe] suggested. Will copy > the relevant comments here to continue the discussion. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13502) Rename/split fs.contract.is-blobstore flag used by contract tests.
[
https://issues.apache.org/jira/browse/HADOOP-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546809#comment-15546809
]
Hadoop QA commented on HADOOP-13502:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
21s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
55s{color} | {color:green} branch-2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
15s{color} | {color:green} branch-2 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
43s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} branch-2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} branch-2 passed with JDK v1.7.0_111 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
8s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
9s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
35s{color} | {color:green} root: The patch generated 0 new + 19 unchanged - 1
fixed = 19 total (was 20) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 47 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
44s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 45s{color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_111. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hadoop-openstack in the patch passed with JDK
v1.7.0_111. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_111.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF Licens
[jira] [Commented] (HADOOP-13685) Document -safely option of rm command.
[
https://issues.apache.org/jira/browse/HADOOP-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546765#comment-15546765
]
Hadoop QA commented on HADOOP-13685:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 9m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HADOOP-13685 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831623/HADOOP-13685.01.patch
|
| Optional Tests | asflicense mvnsite |
| uname | Linux 27a7cb3ec046 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 44f48ee |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10665/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Document -safely option of rm command.
> --
>
> Key: HADOOP-13685
> URL: https://issues.apache.org/jira/browse/HADOOP-13685
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Xiaoyu Yao
>Assignee: Xiaoyu Yao
> Attachments: HADOOP-13685.01.patch
>
>
> HADOOP-12358 introduced -safely option to prevent accidental deletion of
> large directories with lots of files with -rmr -skipTrash. This ticket is
> opened to track document the usage of this feature in
> [FileSystemShell.html|https://hadoop.apache.org/docs/r3.0.0-alpha1/hadoop-project-dist/hadoop-common/FileSystemShell.html#rm].
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13502) Rename/split fs.contract.is-blobstore flag used by contract tests.
[
https://issues.apache.org/jira/browse/HADOOP-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546740#comment-15546740
]
Chris Nauroth commented on HADOOP-13502:
Also, the patch still needs a test run for hadoop-openstack.
> Rename/split fs.contract.is-blobstore flag used by contract tests.
> --
>
> Key: HADOOP-13502
> URL: https://issues.apache.org/jira/browse/HADOOP-13502
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Attachments: HADOOP-13502-branch-2.001.patch,
> HADOOP-13502-branch-2.002.patch, HADOOP-13502-branch-2.003.patch
>
>
> The {{fs.contract.is-blobstore}} flag guards against execution of several
> contract tests to account for known limitations with blob stores. However,
> the name is not entirely accurate, because it's still possible that a file
> system implemented against a blob store could pass those tests, depending on
> whether or not the implementation matches the semantics of HDFS. This issue
> proposes to rename the flag or split it into different flags with different
> definitions for the semantics covered by the current flag.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13502) Rename/split fs.contract.is-blobstore flag used by contract tests.
[
https://issues.apache.org/jira/browse/HADOOP-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546739#comment-15546739
]
Chris Nauroth commented on HADOOP-13502:
The whitespace check triggered warnings on a file I didn't change in this
patch, so it's not relevant.
Xiaoyu, thank you for your review. I'll hold the commit in case Steve has
additional feedback.
> Rename/split fs.contract.is-blobstore flag used by contract tests.
> --
>
> Key: HADOOP-13502
> URL: https://issues.apache.org/jira/browse/HADOOP-13502
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Attachments: HADOOP-13502-branch-2.001.patch,
> HADOOP-13502-branch-2.002.patch, HADOOP-13502-branch-2.003.patch
>
>
> The {{fs.contract.is-blobstore}} flag guards against execution of several
> contract tests to account for known limitations with blob stores. However,
> the name is not entirely accurate, because it's still possible that a file
> system implemented against a blob store could pass those tests, depending on
> whether or not the implementation matches the semantics of HDFS. This issue
> proposes to rename the flag or split it into different flags with different
> definitions for the semantics covered by the current flag.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13685) Document -safely option of rm command.
[ https://issues.apache.org/jira/browse/HADOOP-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HADOOP-13685: Attachment: HADOOP-13685.01.patch > Document -safely option of rm command. > -- > > Key: HADOOP-13685 > URL: https://issues.apache.org/jira/browse/HADOOP-13685 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-13685.01.patch > > > HADOOP-12358 introduced -safely option to prevent accidental deletion of > large directories with lots of files with -rmr -skipTrash. This ticket is > opened to track document the usage of this feature in > [FileSystemShell.html|https://hadoop.apache.org/docs/r3.0.0-alpha1/hadoop-project-dist/hadoop-common/FileSystemShell.html#rm]. > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13685) Document -safely option of rm command.
[ https://issues.apache.org/jira/browse/HADOOP-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HADOOP-13685: Status: Patch Available (was: Open) > Document -safely option of rm command. > -- > > Key: HADOOP-13685 > URL: https://issues.apache.org/jira/browse/HADOOP-13685 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HADOOP-13685.01.patch > > > HADOOP-12358 introduced -safely option to prevent accidental deletion of > large directories with lots of files with -rmr -skipTrash. This ticket is > opened to track document the usage of this feature in > [FileSystemShell.html|https://hadoop.apache.org/docs/r3.0.0-alpha1/hadoop-project-dist/hadoop-common/FileSystemShell.html#rm]. > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13502) Rename/split fs.contract.is-blobstore flag used by contract tests.
[
https://issues.apache.org/jira/browse/HADOOP-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546734#comment-15546734
]
Hadoop QA commented on HADOOP-13502:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
43s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
35s{color} | {color:green} branch-2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
41s{color} | {color:green} branch-2 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
43s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
48s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
51s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} branch-2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} branch-2 passed with JDK v1.7.0_111 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
13s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
35s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 47 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
50s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
55s{color} | {color:green} hadoop-common in the patch passed with JDK
v1.7.0_111. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-openstack in the patch passed with JDK
v1.7.0_111. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_111.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {col
[jira] [Updated] (HADOOP-13685) Document -safely option of rm command.
[ https://issues.apache.org/jira/browse/HADOOP-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HADOOP-13685: Summary: Document -safely option of rm command. (was: Document -safely option to rm command.) > Document -safely option of rm command. > -- > > Key: HADOOP-13685 > URL: https://issues.apache.org/jira/browse/HADOOP-13685 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > > HADOOP-12358 introduced -safely option to prevent accidental deletion of > large directories with lots of files with -rmr -skipTrash. This ticket is > opened to track document the usage of this feature in > [FileSystemShell.html|https://hadoop.apache.org/docs/r3.0.0-alpha1/hadoop-project-dist/hadoop-common/FileSystemShell.html#rm]. > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13685) Document -safely option to rm command.
Xiaoyu Yao created HADOOP-13685: --- Summary: Document -safely option to rm command. Key: HADOOP-13685 URL: https://issues.apache.org/jira/browse/HADOOP-13685 Project: Hadoop Common Issue Type: Improvement Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao HADOOP-12358 introduced -safely option to prevent accidental deletion of large directories with lots of files with -rmr -skipTrash. This ticket is opened to track document the usage of this feature in [FileSystemShell.html|https://hadoop.apache.org/docs/r3.0.0-alpha1/hadoop-project-dist/hadoop-common/FileSystemShell.html#rm]. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13678) Update jackson from 1.9.13 to 2.x in hadoop-tools
[
https://issues.apache.org/jira/browse/HADOOP-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546659#comment-15546659
]
Hadoop QA commented on HADOOP-13678:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green} root: The patch generated 0 new + 322 unchanged - 2
fixed = 322 total (was 324) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
8s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
52s{color} | {color:green} hadoop-yarn-registry in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
25s{color} | {color:green} hadoop-rumen in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hadoop-openstack in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
28s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
0s{color} | {color:green} hadoop-sls in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
24s{color} | {color:green} hadoop-azure-datalake in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 77m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HADOOP-13678 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831600/HADOOP-13678.02.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux bd062ff7c708 3.13.0-93-generic #140-Ubuntu SM
[jira] [Updated] (HADOOP-13684) Snappy may complain Hadoop is built without snappy if libhadoop is not found.
[
https://issues.apache.org/jira/browse/HADOOP-13684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HADOOP-13684:
-
Description:
If for some reason libhadoop can not be found/loaded, Snappy complains Hadoop
is not built with Snappy but it actually is.
{code:title=SnappyCodec.java}
public static void checkNativeCodeLoaded() {
if (!NativeCodeLoader.isNativeCodeLoaded() ||
!NativeCodeLoader.buildSupportsSnappy()) {
throw new RuntimeException("native snappy library not available: " +
"this version of libhadoop was built without " +
"snappy support.");
}
{code}
This case may happen with MAPREDUCE-6577.
was:
If for some reason libhadoop can not be found/loaded, Snappy complains Hadoop
is not built with Snappy but it actually is.
{code:title=SnappyCodec.java}
public static void checkNativeCodeLoaded() {
if (!NativeCodeLoader.isNativeCodeLoaded() ||
!NativeCodeLoader.buildSupportsSnappy()) {
throw new RuntimeException("native snappy library not available: " +
"this version of libhadoop was built without " +
"snappy support.");
}
{code}
> Snappy may complain Hadoop is built without snappy if libhadoop is not found.
> -
>
> Key: HADOOP-13684
> URL: https://issues.apache.org/jira/browse/HADOOP-13684
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Wei-Chiu Chuang
>Priority: Minor
>
> If for some reason libhadoop can not be found/loaded, Snappy complains Hadoop
> is not built with Snappy but it actually is.
> {code:title=SnappyCodec.java}
> public static void checkNativeCodeLoaded() {
> if (!NativeCodeLoader.isNativeCodeLoaded() ||
> !NativeCodeLoader.buildSupportsSnappy()) {
> throw new RuntimeException("native snappy library not available: " +
> "this version of libhadoop was built without " +
> "snappy support.");
> }
> {code}
> This case may happen with MAPREDUCE-6577.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13684) Snappy may complain Hadoop is built without snappy if libhadoop is not found.
Wei-Chiu Chuang created HADOOP-13684:
Summary: Snappy may complain Hadoop is built without snappy if
libhadoop is not found.
Key: HADOOP-13684
URL: https://issues.apache.org/jira/browse/HADOOP-13684
Project: Hadoop Common
Issue Type: Improvement
Reporter: Wei-Chiu Chuang
Priority: Minor
If for some reason libhadoop can not be found/loaded, Snappy complains Hadoop
is not built with Snappy but it actually is.
{code:title=SnappyCodec.java}
public static void checkNativeCodeLoaded() {
if (!NativeCodeLoader.isNativeCodeLoaded() ||
!NativeCodeLoader.buildSupportsSnappy()) {
throw new RuntimeException("native snappy library not available: " +
"this version of libhadoop was built without " +
"snappy support.");
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13502) Rename/split fs.contract.is-blobstore flag used by contract tests.
[
https://issues.apache.org/jira/browse/HADOOP-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546611#comment-15546611
]
Xiaoyu Yao commented on HADOOP-13502:
-
+1 patch 003 pending Jenkins.
> Rename/split fs.contract.is-blobstore flag used by contract tests.
> --
>
> Key: HADOOP-13502
> URL: https://issues.apache.org/jira/browse/HADOOP-13502
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Attachments: HADOOP-13502-branch-2.001.patch,
> HADOOP-13502-branch-2.002.patch, HADOOP-13502-branch-2.003.patch
>
>
> The {{fs.contract.is-blobstore}} flag guards against execution of several
> contract tests to account for known limitations with blob stores. However,
> the name is not entirely accurate, because it's still possible that a file
> system implemented against a blob store could pass those tests, depending on
> whether or not the implementation matches the semantics of HDFS. This issue
> proposes to rename the flag or split it into different flags with different
> definitions for the semantics covered by the current flag.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13502) Rename/split fs.contract.is-blobstore flag used by contract tests.
[
https://issues.apache.org/jira/browse/HADOOP-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-13502:
---
Attachment: HADOOP-13502-branch-2.003.patch
Thank you, Xiaoyu. That's a good suggestion. Here is patch 003.
> Rename/split fs.contract.is-blobstore flag used by contract tests.
> --
>
> Key: HADOOP-13502
> URL: https://issues.apache.org/jira/browse/HADOOP-13502
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Attachments: HADOOP-13502-branch-2.001.patch,
> HADOOP-13502-branch-2.002.patch, HADOOP-13502-branch-2.003.patch
>
>
> The {{fs.contract.is-blobstore}} flag guards against execution of several
> contract tests to account for known limitations with blob stores. However,
> the name is not entirely accurate, because it's still possible that a file
> system implemented against a blob store could pass those tests, depending on
> whether or not the implementation matches the semantics of HDFS. This issue
> proposes to rename the flag or split it into different flags with different
> definitions for the semantics covered by the current flag.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13560) S3ABlockOutputStream to support huge (many GB) file writes
[ https://issues.apache.org/jira/browse/HADOOP-13560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13560: Status: Patch Available (was: Open) tested: s3a ireland > S3ABlockOutputStream to support huge (many GB) file writes > -- > > Key: HADOOP-13560 > URL: https://issues.apache.org/jira/browse/HADOOP-13560 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.9.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Attachments: HADOOP-13560-branch-2-001.patch, > HADOOP-13560-branch-2-002.patch, HADOOP-13560-branch-2-003.patch, > HADOOP-13560-branch-2-004.patch > > > An AWS SDK [issue|https://github.com/aws/aws-sdk-java/issues/367] highlights > that metadata isn't copied on large copies. > 1. Add a test to do that large copy/rname and verify that the copy really > works > 2. Verify that metadata makes it over. > Verifying large file rename is important on its own, as it is needed for very > large commit operations for committers using rename -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13560) S3ABlockOutputStream to support huge (many GB) file writes
[
https://issues.apache.org/jira/browse/HADOOP-13560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546540#comment-15546540
]
Steve Loughran commented on HADOOP-13560:
-
Latest patch: factored out semaphore queue in front of an ExecutorService into
{{SemaphoredDelegatingExecutor}}. As well as being used to manage thread pool
load, each block output stream gets its own submissions to the central thread
pool limited by the property {{fs.s3a.block.output.active.limit}}. This will
let us have a larger common pool, but still limit the amount of bandwidth a
single stream can consume. Which I need as I want to use that same thread pool
for parallel rename operations and async mkdir/delete dir operations.
> S3ABlockOutputStream to support huge (many GB) file writes
> --
>
> Key: HADOOP-13560
> URL: https://issues.apache.org/jira/browse/HADOOP-13560
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13560-branch-2-001.patch,
> HADOOP-13560-branch-2-002.patch, HADOOP-13560-branch-2-003.patch,
> HADOOP-13560-branch-2-004.patch
>
>
> An AWS SDK [issue|https://github.com/aws/aws-sdk-java/issues/367] highlights
> that metadata isn't copied on large copies.
> 1. Add a test to do that large copy/rname and verify that the copy really
> works
> 2. Verify that metadata makes it over.
> Verifying large file rename is important on its own, as it is needed for very
> large commit operations for committers using rename
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13681) Reduce Kafka dependencies in hadoop-kafka module
[
https://issues.apache.org/jira/browse/HADOOP-13681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-13681:
---
Assignee: Grant Henke
> Reduce Kafka dependencies in hadoop-kafka module
>
>
> Key: HADOOP-13681
> URL: https://issues.apache.org/jira/browse/HADOOP-13681
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 3.0.0-alpha1
>Reporter: Grant Henke
>Assignee: Grant Henke
> Attachments: HADOOP-13681-1.patch
>
>
> The newly added Kafka module defines the Kafka dependency as:
> {noformat}
> ...
> org.apache.kafka
> kafka-2.10
> ${kafka.version}
> ...
> {noformat}
> This is unfavorable because its using the server dependency, which
> transitively has the client jars. The server dependency includes all of the
> server code and some larger transitive dependencies like Scala and Zookeeper.
> Instead the pom file should be changed to only depend on the clients jar
> which is a much smaller footprint:
> {noformat}
> ...
> org.apache.kafka
> kafka-clients
> ${kafka.version}
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13681) Reduce Kafka dependencies in hadoop-kafka module
[
https://issues.apache.org/jira/browse/HADOOP-13681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546532#comment-15546532
]
Akira Ajisaka commented on HADOOP-13681:
Nice catch! LGTM, +1.
> Reduce Kafka dependencies in hadoop-kafka module
>
>
> Key: HADOOP-13681
> URL: https://issues.apache.org/jira/browse/HADOOP-13681
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 3.0.0-alpha1
>Reporter: Grant Henke
> Attachments: HADOOP-13681-1.patch
>
>
> The newly added Kafka module defines the Kafka dependency as:
> {noformat}
> ...
> org.apache.kafka
> kafka-2.10
> ${kafka.version}
> ...
> {noformat}
> This is unfavorable because its using the server dependency, which
> transitively has the client jars. The server dependency includes all of the
> server code and some larger transitive dependencies like Scala and Zookeeper.
> Instead the pom file should be changed to only depend on the clients jar
> which is a much smaller footprint:
> {noformat}
> ...
> org.apache.kafka
> kafka-clients
> ${kafka.version}
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12977) s3a ignores delete("/", true)
[
https://issues.apache.org/jira/browse/HADOOP-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546525#comment-15546525
]
Chris Nauroth commented on HADOOP-12977:
I'm not sure a test for "//" adds much value, considering that when the path
string gets passed through {{Path}} / {{URI}}, the consecutive '/' characters
will get normalized down to just one '/'. There is no harm in testing it, but
I think it just degenerates to the same thing as testing "/". This would
really be more of a unit test of {{Path}} behavior than {{FileSystem}}
behavior. Actually, I just checked and that's covered by a unit test in
{{TestPath#testNormalize}}.
> s3a ignores delete("/", true)
> -
>
> Key: HADOOP-12977
> URL: https://issues.apache.org/jira/browse/HADOOP-12977
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-12977-001.patch, HADOOP-12977-002.patch,
> HADOOP-12977-branch-2-002.patch, HADOOP-12977-branch-2-002.patch
>
>
> if you try to delete the root directory on s3a, you get politely but firmly
> told you can't
> {code}
> 2016-03-30 12:01:44,924 INFO s3a.S3AFileSystem
> (S3AFileSystem.java:delete(638)) - s3a cannot delete the root directory
> {code}
> The semantics of {{rm -rf "/"}} are defined, they are "delete everything
> underneath, while preserving the root dir itself".
> # s3a needs to support this.
> # this skipped through the FS contract tests in
> {{AbstractContractRootDirectoryTest}}; the option of whether deleting / works
> or not should be made configurable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13502) Rename/split fs.contract.is-blobstore flag used by contract tests.
[
https://issues.apache.org/jira/browse/HADOOP-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546524#comment-15546524
]
Xiaoyu Yao commented on HADOOP-13502:
-
Thanks [~cnauroth] for the update. Just one NIT:
in AbstractContractCreateTest.java, can we remove the assumption of *object
store* in the comments and the skip message to match the description of the new
flag, which should be applicable to all HCFS?
{code}
// object store: downgrade to a skip so that the failure is visible
// in test results
skip("Filesystem is an object store and newly created files are not
immediately visible");
{code}
> Rename/split fs.contract.is-blobstore flag used by contract tests.
> --
>
> Key: HADOOP-13502
> URL: https://issues.apache.org/jira/browse/HADOOP-13502
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Attachments: HADOOP-13502-branch-2.001.patch,
> HADOOP-13502-branch-2.002.patch
>
>
> The {{fs.contract.is-blobstore}} flag guards against execution of several
> contract tests to account for known limitations with blob stores. However,
> the name is not entirely accurate, because it's still possible that a file
> system implemented against a blob store could pass those tests, depending on
> whether or not the implementation matches the semantics of HDFS. This issue
> proposes to rename the flag or split it into different flags with different
> definitions for the semantics covered by the current flag.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13560) S3ABlockOutputStream to support huge (many GB) file writes
[
https://issues.apache.org/jira/browse/HADOOP-13560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546509#comment-15546509
]
Steve Loughran commented on HADOOP-13560:
-
think I've addressed most of them. Realised that commenting in github makes
replying to them one by one easily.
anyway, to most of the anwers are "yes, you are write, fixed"
one exception, why {{ should DiskBlock#startUpload wrap the returned stream in
a BufferedInputStream)}}.
no, because if there is a transient PUT failure and AWS tries to replay the
PUT, it can't roll back the buffered input stream back to the beginning of the
file.; only back the beginning of the last buffered area. This is something
that surfaced in the huge file tests.
> S3ABlockOutputStream to support huge (many GB) file writes
> --
>
> Key: HADOOP-13560
> URL: https://issues.apache.org/jira/browse/HADOOP-13560
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13560-branch-2-001.patch,
> HADOOP-13560-branch-2-002.patch, HADOOP-13560-branch-2-003.patch,
> HADOOP-13560-branch-2-004.patch
>
>
> An AWS SDK [issue|https://github.com/aws/aws-sdk-java/issues/367] highlights
> that metadata isn't copied on large copies.
> 1. Add a test to do that large copy/rname and verify that the copy really
> works
> 2. Verify that metadata makes it over.
> Verifying large file rename is important on its own, as it is needed for very
> large commit operations for committers using rename
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13522) Permissions on fs stat added
[ https://issues.apache.org/jira/browse/HADOOP-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Garbarini updated HADOOP-13522: Assignee: Akira Ajisaka (was: Alex Garbarini) > Permissions on fs stat added > > > Key: HADOOP-13522 > URL: https://issues.apache.org/jira/browse/HADOOP-13522 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.0.0-alpha1 >Reporter: Alex Garbarini >Assignee: Akira Ajisaka > Attachments: HADOOP-13522.001.patch, HADOOP-13522.002.patch, > HADOOP-13522.003.patch, HADOOP-13522.004.patch, HADOOP-13522.005.patch, > HADOOP-13522.006.patch > > > This patch adds to fs/shell/Stat.java the missing options of %a and %A. > FileStatus already contains the getPermission() method required for returning > symbolic permissions. FsPermission contains the method to return the binary > short, but nothing to present in standard Octal format. > Most UNIX admins base their work on such standard octal permissions. Hence, > this patch also introduces one tiny method to translate the toShort() return > into octal. > Build has already passed unit tests and javadoc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13522) Permissions on fs stat added
[ https://issues.apache.org/jira/browse/HADOOP-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Garbarini updated HADOOP-13522: Status: Patch Available (was: Open) > Permissions on fs stat added > > > Key: HADOOP-13522 > URL: https://issues.apache.org/jira/browse/HADOOP-13522 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.0.0-alpha1 >Reporter: Alex Garbarini >Assignee: Alex Garbarini > Attachments: HADOOP-13522.001.patch, HADOOP-13522.002.patch, > HADOOP-13522.003.patch, HADOOP-13522.004.patch, HADOOP-13522.005.patch, > HADOOP-13522.006.patch > > > This patch adds to fs/shell/Stat.java the missing options of %a and %A. > FileStatus already contains the getPermission() method required for returning > symbolic permissions. FsPermission contains the method to return the binary > short, but nothing to present in standard Octal format. > Most UNIX admins base their work on such standard octal permissions. Hence, > this patch also introduces one tiny method to translate the toShort() return > into octal. > Build has already passed unit tests and javadoc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13522) Permissions on fs stat added
[ https://issues.apache.org/jira/browse/HADOOP-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Garbarini updated HADOOP-13522: Attachment: HADOOP-13522.006.patch Rebased the patch for the latest trunk. > Permissions on fs stat added > > > Key: HADOOP-13522 > URL: https://issues.apache.org/jira/browse/HADOOP-13522 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.0.0-alpha1 >Reporter: Alex Garbarini >Assignee: Alex Garbarini > Attachments: HADOOP-13522.001.patch, HADOOP-13522.002.patch, > HADOOP-13522.003.patch, HADOOP-13522.004.patch, HADOOP-13522.005.patch, > HADOOP-13522.006.patch > > > This patch adds to fs/shell/Stat.java the missing options of %a and %A. > FileStatus already contains the getPermission() method required for returning > symbolic permissions. FsPermission contains the method to return the binary > short, but nothing to present in standard Octal format. > Most UNIX admins base their work on such standard octal permissions. Hence, > this patch also introduces one tiny method to translate the toShort() return > into octal. > Build has already passed unit tests and javadoc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13683) Update jackson from 1.9.13 to 2.x in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HADOOP-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13683: --- Hadoop Flags: Incompatible change Release Note: Removed jackson 1.9.13 dependency from hadoop-hdfs-project module. Status: Patch Available (was: Open) > Update jackson from 1.9.13 to 2.x in hadoop-hdfs > > > Key: HADOOP-13683 > URL: https://issues.apache.org/jira/browse/HADOOP-13683 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-13683.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13683) Update jackson from 1.9.13 to 2.x in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HADOOP-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13683: --- Attachment: HADOOP-13683.01.patch > Update jackson from 1.9.13 to 2.x in hadoop-hdfs > > > Key: HADOOP-13683 > URL: https://issues.apache.org/jira/browse/HADOOP-13683 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-13683.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13614) Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
[
https://issues.apache.org/jira/browse/HADOOP-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546454#comment-15546454
]
Chris Nauroth commented on HADOOP-13614:
Oh, but pre-commit is going to the GitHub pull request. I guess you'll need to
update it there, but that's easily done with the 005 file.
> Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
> ---
>
> Key: HADOOP-13614
> URL: https://issues.apache.org/jira/browse/HADOOP-13614
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13614-branch-2-001.patch,
> HADOOP-13614-branch-2-002.patch, HADOOP-13614-branch-2-002.patch,
> HADOOP-13614-branch-2-004.patch, HADOOP-13614-branch-2-005.patch, testrun.txt
>
>
> Some of the slow test cases contain tests that are now obsoleted by newer
> ones. For example, {{ITestS3ADeleteManyFiles}} has the test case
> {{testOpenCreate()}} which writes then reads files up 25 MB.
> Have a look at which of the s3a tests are taking time, review them to see if
> newer tests have superceded the slow ones; and cut them where appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13614) Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
[
https://issues.apache.org/jira/browse/HADOOP-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546450#comment-15546450
]
Hadoop QA commented on HADOOP-13614:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color}
| {color:red} HADOOP-13614 does not apply to branch-2. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-13614 |
| GITHUB PR | https://github.com/apache/hadoop/pull/132 |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10659/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
> ---
>
> Key: HADOOP-13614
> URL: https://issues.apache.org/jira/browse/HADOOP-13614
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13614-branch-2-001.patch,
> HADOOP-13614-branch-2-002.patch, HADOOP-13614-branch-2-002.patch,
> HADOOP-13614-branch-2-004.patch, HADOOP-13614-branch-2-005.patch, testrun.txt
>
>
> Some of the slow test cases contain tests that are now obsoleted by newer
> ones. For example, {{ITestS3ADeleteManyFiles}} has the test case
> {{testOpenCreate()}} which writes then reads files up 25 MB.
> Have a look at which of the s3a tests are taking time, review them to see if
> newer tests have superceded the slow ones; and cut them where appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13614) Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
[
https://issues.apache.org/jira/browse/HADOOP-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-13614:
---
Attachment: HADOOP-13614-branch-2-005.patch
[~ste...@apache.org], when I committed my HADOOP-13674 patch, I caused a
conflict for your patch here. I rebased it for you, and I'm attaching it as
revision 005. The only change since 004 is resolving some conflicts on import
statements in {{ITestS3AFailureHandling}}. This version of the patch passes
tests again US-west-2 for me, and I'm code reviewing in more detail now.
> Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
> ---
>
> Key: HADOOP-13614
> URL: https://issues.apache.org/jira/browse/HADOOP-13614
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13614-branch-2-001.patch,
> HADOOP-13614-branch-2-002.patch, HADOOP-13614-branch-2-002.patch,
> HADOOP-13614-branch-2-004.patch, HADOOP-13614-branch-2-005.patch, testrun.txt
>
>
> Some of the slow test cases contain tests that are now obsoleted by newer
> ones. For example, {{ITestS3ADeleteManyFiles}} has the test case
> {{testOpenCreate()}} which writes then reads files up 25 MB.
> Have a look at which of the s3a tests are taking time, review them to see if
> newer tests have superceded the slow ones; and cut them where appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-13683) Update jackson from 1.9.13 to 2.x in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HADOOP-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-13683: -- Assignee: Akira Ajisaka > Update jackson from 1.9.13 to 2.x in hadoop-hdfs > > > Key: HADOOP-13683 > URL: https://issues.apache.org/jira/browse/HADOOP-13683 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13683) Update jackson from 1.9.13 to 2.x in hadoop-hdfs
Akira Ajisaka created HADOOP-13683: -- Summary: Update jackson from 1.9.13 to 2.x in hadoop-hdfs Key: HADOOP-13683 URL: https://issues.apache.org/jira/browse/HADOOP-13683 Project: Hadoop Common Issue Type: Sub-task Reporter: Akira Ajisaka -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13502) Rename/split fs.contract.is-blobstore flag used by contract tests.
[
https://issues.apache.org/jira/browse/HADOOP-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-13502:
---
Attachment: HADOOP-13502-branch-2.002.patch
Steve and I spoke offline. I had a misunderstanding about his last comment.
His point was that we really don't have any other known object stores. There
are other file system implementations, but as far as we know, they are not
object stores, and therefore they would never set the is-blobstore flag.
Based on that, I'm uploading patch 002. This retains is-blobstore as an
informational flag. It is not deprecated. The tests no longer need to check
for is-blobstore, because all known object stores have had their contract XML
options updated. As a side effect, the patch no longer needs to touch wasb.xml.
> Rename/split fs.contract.is-blobstore flag used by contract tests.
> --
>
> Key: HADOOP-13502
> URL: https://issues.apache.org/jira/browse/HADOOP-13502
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Attachments: HADOOP-13502-branch-2.001.patch,
> HADOOP-13502-branch-2.002.patch
>
>
> The {{fs.contract.is-blobstore}} flag guards against execution of several
> contract tests to account for known limitations with blob stores. However,
> the name is not entirely accurate, because it's still possible that a file
> system implemented against a blob store could pass those tests, depending on
> whether or not the implementation matches the semantics of HDFS. This issue
> proposes to rename the flag or split it into different flags with different
> definitions for the semantics covered by the current flag.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13678) Update jackson from 1.9.13 to 2.x in hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13678: --- Attachment: HADOOP-13678.02.patch 02 patch: * Removed trailing whitespaces * Fixes test failure > Update jackson from 1.9.13 to 2.x in hadoop-tools > - > > Key: HADOOP-13678 > URL: https://issues.apache.org/jira/browse/HADOOP-13678 > Project: Hadoop Common > Issue Type: Sub-task > Components: tools >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-13678.01.patch, HADOOP-13678.02.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-10101) Update guava dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546286#comment-15546286 ] Hitesh Shah commented on HADOOP-10101: -- I think it is high time we bump up the dependency however we should look to hiding it too i.e. shading it as needed so that downstream users can use whatever version of guava they want to use. Some questions: - why version 18 and not 19? Guava 19 was released in 2015 or so - have you confirmed that no deprecated and/or beta apis are being used? - I believe there are some places where guava objects are being used in public facing APIs. This might not be something to address in this jira but should definitely be looked at as a blocker before the next release in addition to my initial comment. > Update guava dependency to the latest version > - > > Key: HADOOP-10101 > URL: https://issues.apache.org/jira/browse/HADOOP-10101 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.2.0, 2.6.0 >Reporter: Rakesh R >Assignee: Vinayakumar B > Labels: BB2015-05-TBR > Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, > HADOOP-10101-005.patch, HADOOP-10101-006.patch, HADOOP-10101-007.patch, > HADOOP-10101-008.patch, HADOOP-10101-009.patch, HADOOP-10101-009.patch, > HADOOP-10101-010.patch, HADOOP-10101-010.patch, HADOOP-10101-011.patch, > HADOOP-10101.patch, HADOOP-10101.patch > > > The existing guava version is 11.0.2 which is quite old. This -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated HADOOP-10101: - Target Version/s: 3.0.0-alpha2 (was: ) > Update guava dependency to the latest version > - > > Key: HADOOP-10101 > URL: https://issues.apache.org/jira/browse/HADOOP-10101 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.2.0, 2.6.0 >Reporter: Rakesh R >Assignee: Vinayakumar B > Labels: BB2015-05-TBR > Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, > HADOOP-10101-005.patch, HADOOP-10101-006.patch, HADOOP-10101-007.patch, > HADOOP-10101-008.patch, HADOOP-10101-009.patch, HADOOP-10101-009.patch, > HADOOP-10101-010.patch, HADOOP-10101-010.patch, HADOOP-10101-011.patch, > HADOOP-10101.patch, HADOOP-10101.patch > > > The existing guava version is 11.0.2 which is quite old. This -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13674) S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint.
[ https://issues.apache.org/jira/browse/HADOOP-13674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546152#comment-15546152 ] Hudson commented on HADOOP-13674: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10540 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10540/]) HADOOP-13674. S3A can provide a more detailed error message when (cnauroth: rev 88b9444a81081da9b168d2e290f9552b58a4d8c6) * (add) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AExceptionTranslation.java * (edit) hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md * (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFailureHandling.java > S3A can provide a more detailed error message when accessing a bucket through > an incorrect S3 endpoint. > --- > > Key: HADOOP-13674 > URL: https://issues.apache.org/jira/browse/HADOOP-13674 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Fix For: 2.8.0 > > Attachments: HADOOP-13674-branch-2.001.patch, > HADOOP-13674-branch-2.002.patch > > > When accessing the S3 service through a region-specific endpoint, the bucket > must be located in that region. If the client attempts to access a bucket > that is not located in that region, then the service replies with a 301 > redirect and the correct region endpoint. However, the exception thrown by > S3A does not include the correct endpoint. If we included that information > in the exception, it would make it easier for users to diagnose and fix > incorrect configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13674) S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint.
[ https://issues.apache.org/jira/browse/HADOOP-13674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-13674: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) Pre-commit has given a +1, and I have confirmed a full test run passes against US-west-2. I have committed this to trunk, branch-2 and branch-2.8. Steve, thank you for the code review. > S3A can provide a more detailed error message when accessing a bucket through > an incorrect S3 endpoint. > --- > > Key: HADOOP-13674 > URL: https://issues.apache.org/jira/browse/HADOOP-13674 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Fix For: 2.8.0 > > Attachments: HADOOP-13674-branch-2.001.patch, > HADOOP-13674-branch-2.002.patch > > > When accessing the S3 service through a region-specific endpoint, the bucket > must be located in that region. If the client attempts to access a bucket > that is not located in that region, then the service replies with a 301 > redirect and the correct region endpoint. However, the exception thrown by > S3A does not include the correct endpoint. If we included that information > in the exception, it would make it easier for users to diagnose and fix > incorrect configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13682) Log missing stacktrace in catch block
[ https://issues.apache.org/jira/browse/HADOOP-13682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546042#comment-15546042 ] Wei-Chiu Chuang commented on HADOOP-13682: -- Good point. I would argue that debug logs need stacktrace but not info logs. > Log missing stacktrace in catch block > - > > Key: HADOOP-13682 > URL: https://issues.apache.org/jira/browse/HADOOP-13682 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Priority: Minor > Attachments: common_output > > > Wrote a simple [code parser|https://github.com/jojochuang/codebetter] to find > all instances of exception objects that are caught but logged without stack > trace. Some of them may have legitimate reason not to log stack trace, but > other may worth an improvement. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13682) Log missing stacktrace in catch block
[ https://issues.apache.org/jira/browse/HADOOP-13682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546035#comment-15546035 ] Allen Wittenauer commented on HADOOP-13682: --- This sounds like it is going in the opposite direction. In order to make Hadoop user friendly, we should be eliminating stack traces, not generating more. > Log missing stacktrace in catch block > - > > Key: HADOOP-13682 > URL: https://issues.apache.org/jira/browse/HADOOP-13682 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Priority: Minor > Attachments: common_output > > > Wrote a simple [code parser|https://github.com/jojochuang/codebetter] to find > all instances of exception objects that are caught but logged without stack > trace. Some of them may have legitimate reason not to log stack trace, but > other may worth an improvement. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13682) Log missing stacktrace in catch block
Wei-Chiu Chuang created HADOOP-13682: Summary: Log missing stacktrace in catch block Key: HADOOP-13682 URL: https://issues.apache.org/jira/browse/HADOOP-13682 Project: Hadoop Common Issue Type: Improvement Reporter: Wei-Chiu Chuang Priority: Minor Attachments: common_output Wrote a simple [code parser|https://github.com/jojochuang/codebetter] to find all instances of exception objects that are caught but logged without stack trace. Some of them may have legitimate reason not to log stack trace, but other may worth an improvement. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13682) Log missing stacktrace in catch block
[ https://issues.apache.org/jira/browse/HADOOP-13682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-13682: - Attachment: common_output > Log missing stacktrace in catch block > - > > Key: HADOOP-13682 > URL: https://issues.apache.org/jira/browse/HADOOP-13682 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Priority: Minor > Attachments: common_output > > > Wrote a simple [code parser|https://github.com/jojochuang/codebetter] to find > all instances of exception objects that are caught but logged without stack > trace. Some of them may have legitimate reason not to log stack trace, but > other may worth an improvement. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13614) Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
[
https://issues.apache.org/jira/browse/HADOOP-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546019#comment-15546019
]
Hadoop QA commented on HADOOP-13614:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m
32s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 22 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
44s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
51s{color} | {color:green} branch-2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
37s{color} | {color:green} branch-2 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
23s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
26s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} branch-2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} branch-2 passed with JDK v1.7.0_111 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
52s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 5m 52s{color}
| {color:red} root-jdk1.8.0_101 with JDK v1.8.0_101 generated 1 new + 857
unchanged - 0 fixed = 858 total (was 857) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
34s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 6m 34s{color}
| {color:red} root-jdk1.7.0_111 with JDK v1.7.0_111 generated 1 new + 950
unchanged - 0 fixed = 951 total (was 950) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 27s{color} | {color:orange} root: The patch generated 4 new + 39 unchanged -
1 fixed = 43 total (was 40) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 47 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
0s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
49s{color} | {color:green} hadoop-common in the patch passed with JDK
v1.7.0_111. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_111.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License
[jira] [Commented] (HADOOP-13674) S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint.
[
https://issues.apache.org/jira/browse/HADOOP-13674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546009#comment-15546009
]
Hadoop QA commented on HADOOP-13674:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
17s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} branch-2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} branch-2 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} branch-2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} branch-2 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
20s{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_111.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:b59b8b7 |
| JIRA Issue | HADOOP-13674 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831567/HADOOP-13674-branch-2.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ee7f904dd68c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2 / a9a2f64 |
| Default Java | 1.7.0_111 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_101
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 |
| findbugs | v3.0.0 |
| JDK v1.7.0_11
[jira] [Updated] (HADOOP-13674) S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint.
[ https://issues.apache.org/jira/browse/HADOOP-13674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-13674: --- Attachment: HADOOP-13674-branch-2.002.patch Here is patch 002 cleaning up the Checkstyle and whitespace warnings. I'll wait for a pre-commit run and complete a full test pass against the S3 service. Then, I'll commit based on Steve's +1. > S3A can provide a more detailed error message when accessing a bucket through > an incorrect S3 endpoint. > --- > > Key: HADOOP-13674 > URL: https://issues.apache.org/jira/browse/HADOOP-13674 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Attachments: HADOOP-13674-branch-2.001.patch, > HADOOP-13674-branch-2.002.patch > > > When accessing the S3 service through a region-specific endpoint, the bucket > must be located in that region. If the client attempts to access a bucket > that is not located in that region, then the service replies with a 301 > redirect and the correct region endpoint. However, the exception thrown by > S3A does not include the correct endpoint. If we included that information > in the exception, it would make it easier for users to diagnose and fix > incorrect configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13681) Reduce Kafka dependencies in hadoop-kafka module
[
https://issues.apache.org/jira/browse/HADOOP-13681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545913#comment-15545913
]
Grant Henke commented on HADOOP-13681:
--
No added tests since this one line change is just reducing the dependencies.
> Reduce Kafka dependencies in hadoop-kafka module
>
>
> Key: HADOOP-13681
> URL: https://issues.apache.org/jira/browse/HADOOP-13681
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 3.0.0-alpha1
>Reporter: Grant Henke
> Attachments: HADOOP-13681-1.patch
>
>
> The newly added Kafka module defines the Kafka dependency as:
> {noformat}
> ...
> org.apache.kafka
> kafka-2.10
> ${kafka.version}
> ...
> {noformat}
> This is unfavorable because its using the server dependency, which
> transitively has the client jars. The server dependency includes all of the
> server code and some larger transitive dependencies like Scala and Zookeeper.
> Instead the pom file should be changed to only depend on the clients jar
> which is a much smaller footprint:
> {noformat}
> ...
> org.apache.kafka
> kafka-clients
> ${kafka.version}
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13681) Reduce Kafka dependencies in hadoop-kafka module
[
https://issues.apache.org/jira/browse/HADOOP-13681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545866#comment-15545866
]
Hadoop QA commented on HADOOP-13681:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
10s{color} | {color:green} hadoop-kafka in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HADOOP-13681 |
| GITHUB PR | https://github.com/apache/hadoop/pull/133 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml |
| uname | Linux 73719c6ce8f8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ef7f06f |
| Default Java | 1.8.0_101 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10657/testReport/ |
| modules | C: hadoop-tools/hadoop-kafka U: hadoop-tools/hadoop-kafka |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10657/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Reduce Kafka dependencies in hadoop-kafka module
>
>
> Key: HADOOP-13681
> URL: https://issues.apache.org/jira/browse/HADOOP-13681
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 3.0.0-alpha1
>Reporter: Grant Henke
> Attachments: HADOOP-13681-1.patch
>
>
> The newly added Kafka module defines the Kafka dependency as:
> {noformat}
> ...
> org.apache.kafka
> kafka-2.10
> ${kafka.version}
> ...
> {noformat}
> This is unfavorable because its using the server dependency, which
> transitively has the client jars. The server dependency includes all of the
> server code and some larger transitive dependencies like Scala and Zookeeper.
> Instead the pom file should be changed to only depend on the clients jar
> which is a much smaller footprint:
> {noformat}
> ...
> org.ap
[jira] [Updated] (HADOOP-13681) Reduce Kafka dependencies in hadoop-kafka module
[
https://issues.apache.org/jira/browse/HADOOP-13681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated HADOOP-13681:
-
Attachment: HADOOP-13681-1.patch
> Reduce Kafka dependencies in hadoop-kafka module
>
>
> Key: HADOOP-13681
> URL: https://issues.apache.org/jira/browse/HADOOP-13681
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 3.0.0-alpha1
>Reporter: Grant Henke
> Attachments: HADOOP-13681-1.patch
>
>
> The newly added Kafka module defines the Kafka dependency as:
> {noformat}
> ...
> org.apache.kafka
> kafka-2.10
> ${kafka.version}
> ...
> {noformat}
> This is unfavorable because its using the server dependency, which
> transitively has the client jars. The server dependency includes all of the
> server code and some larger transitive dependencies like Scala and Zookeeper.
> Instead the pom file should be changed to only depend on the clients jar
> which is a much smaller footprint:
> {noformat}
> ...
> org.apache.kafka
> kafka-clients
> ${kafka.version}
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13681) Reduce Kafka dependencies in hadoop-kafka module
[
https://issues.apache.org/jira/browse/HADOOP-13681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated HADOOP-13681:
-
Status: Patch Available (was: Open)
> Reduce Kafka dependencies in hadoop-kafka module
>
>
> Key: HADOOP-13681
> URL: https://issues.apache.org/jira/browse/HADOOP-13681
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 3.0.0-alpha1
>Reporter: Grant Henke
>
> The newly added Kafka module defines the Kafka dependency as:
> {noformat}
> ...
> org.apache.kafka
> kafka-2.10
> ${kafka.version}
> ...
> {noformat}
> This is unfavorable because its using the server dependency, which
> transitively has the client jars. The server dependency includes all of the
> server code and some larger transitive dependencies like Scala and Zookeeper.
> Instead the pom file should be changed to only depend on the clients jar
> which is a much smaller footprint:
> {noformat}
> ...
> org.apache.kafka
> kafka-clients
> ${kafka.version}
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13681) Reduce Kafka dependencies in hadoop-kafka module
[
https://issues.apache.org/jira/browse/HADOOP-13681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545762#comment-15545762
]
ASF GitHub Bot commented on HADOOP-13681:
-
GitHub user granthenke opened a pull request:
https://github.com/apache/hadoop/pull/133
HADOOP-13681. Reduce Kafka dependencies in hadoop-kafka module
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/granthenke/hadoop kafka-deps
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hadoop/pull/133.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #133
commit b2b738b362f7a8db1f80892e148ff6180bc901c4
Author: Grant Henke
Date: 2016-10-04T15:34:27Z
HADOOP-13681. Reduce Kafka dependencies in hadoop-kafka module
> Reduce Kafka dependencies in hadoop-kafka module
>
>
> Key: HADOOP-13681
> URL: https://issues.apache.org/jira/browse/HADOOP-13681
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 3.0.0-alpha1
>Reporter: Grant Henke
>
> The newly added Kafka module defines the Kafka dependency as:
> {noformat}
> ...
> org.apache.kafka
> kafka-2.10
> ${kafka.version}
> ...
> {noformat}
> This is unfavorable because its using the server dependency, which
> transitively has the client jars. The server dependency includes all of the
> server code and some larger transitive dependencies like Scala and Zookeeper.
> Instead the pom file should be changed to only depend on the clients jar
> which is a much smaller footprint:
> {noformat}
> ...
> org.apache.kafka
> kafka-clients
> ${kafka.version}
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13681) Reduce Kafka dependencies in hadoop-kafka module
[
https://issues.apache.org/jira/browse/HADOOP-13681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545664#comment-15545664
]
Grant Henke commented on HADOOP-13681:
--
I will provide a simple patch shortly
> Reduce Kafka dependencies in hadoop-kafka module
>
>
> Key: HADOOP-13681
> URL: https://issues.apache.org/jira/browse/HADOOP-13681
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 3.0.0-alpha1
>Reporter: Grant Henke
>
> The newly added Kafka module defines the Kafka dependency as:
> {noformat}
> ...
> org.apache.kafka
> kafka-2.10
> ${kafka.version}
> ...
> {noformat}
> This is unfavorable because its using the server dependency, which
> transitively has the client jars. The server dependency includes all of the
> server code and some larger transitive dependencies like Scala and Zookeeper.
> Instead the pom file should be changed to only depend on the clients jar
> which is a much smaller footprint:
> {noformat}
> ...
> org.apache.kafka
> kafka-clients
> ${kafka.version}
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13681) Reduce Kafka dependencies in hadoop-kafka module
Grant Henke created HADOOP-13681:
Summary: Reduce Kafka dependencies in hadoop-kafka module
Key: HADOOP-13681
URL: https://issues.apache.org/jira/browse/HADOOP-13681
Project: Hadoop Common
Issue Type: Improvement
Components: metrics
Affects Versions: 3.0.0-alpha1
Reporter: Grant Henke
The newly added Kafka module defines the Kafka dependency as:
{noformat}
...
org.apache.kafka
kafka-2.10
${kafka.version}
...
{noformat}
This is unfavorable because its using the server dependency, which transitively
has the client jars. The server dependency includes all of the server code and
some larger transitive dependencies like Scala and Zookeeper.
Instead the pom file should be changed to only depend on the clients jar which
is a much smaller footprint:
{noformat}
...
org.apache.kafka
kafka-clients
${kafka.version}
...
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13560) S3ABlockOutputStream to support huge (many GB) file writes
[ https://issues.apache.org/jira/browse/HADOOP-13560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13560: Status: Open (was: Patch Available) > S3ABlockOutputStream to support huge (many GB) file writes > -- > > Key: HADOOP-13560 > URL: https://issues.apache.org/jira/browse/HADOOP-13560 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.9.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Attachments: HADOOP-13560-branch-2-001.patch, > HADOOP-13560-branch-2-002.patch, HADOOP-13560-branch-2-003.patch, > HADOOP-13560-branch-2-004.patch > > > An AWS SDK [issue|https://github.com/aws/aws-sdk-java/issues/367] highlights > that metadata isn't copied on large copies. > 1. Add a test to do that large copy/rname and verify that the copy really > works > 2. Verify that metadata makes it over. > Verifying large file rename is important on its own, as it is needed for very > large commit operations for committers using rename -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13614) Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
[
https://issues.apache.org/jira/browse/HADOOP-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13614:
Attachment: HADOOP-13614-branch-2-004.patch
patch 004; timeouts for all tests fixed to 10 min, scale tests (and distcp
contract) set to a scale value of 30 min.
Tests all worked against s3 ireland.
Interesting that Chris saw timeouts against US east and I didn't see them
against US ireland, makes me wonder if different endpoints have different
latencies
> Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
> ---
>
> Key: HADOOP-13614
> URL: https://issues.apache.org/jira/browse/HADOOP-13614
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13614-branch-2-001.patch,
> HADOOP-13614-branch-2-002.patch, HADOOP-13614-branch-2-002.patch,
> HADOOP-13614-branch-2-004.patch, testrun.txt
>
>
> Some of the slow test cases contain tests that are now obsoleted by newer
> ones. For example, {{ITestS3ADeleteManyFiles}} has the test case
> {{testOpenCreate()}} which writes then reads files up 25 MB.
> Have a look at which of the s3a tests are taking time, review them to see if
> newer tests have superceded the slow ones; and cut them where appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13614) Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
[
https://issues.apache.org/jira/browse/HADOOP-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13614:
Status: Patch Available (was: Open)
> Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
> ---
>
> Key: HADOOP-13614
> URL: https://issues.apache.org/jira/browse/HADOOP-13614
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-13614-branch-2-001.patch,
> HADOOP-13614-branch-2-002.patch, HADOOP-13614-branch-2-002.patch,
> HADOOP-13614-branch-2-004.patch, testrun.txt
>
>
> Some of the slow test cases contain tests that are now obsoleted by newer
> ones. For example, {{ITestS3ADeleteManyFiles}} has the test case
> {{testOpenCreate()}} which writes then reads files up 25 MB.
> Have a look at which of the s3a tests are taking time, review them to see if
> newer tests have superceded the slow ones; and cut them where appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13678) Update jackson from 1.9.13 to 2.x in hadoop-tools
[
https://issues.apache.org/jira/browse/HADOOP-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545571#comment-15545571
]
Hadoop QA commented on HADOOP-13678:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
35s{color} | {color:green} root: The patch generated 0 new + 322 unchanged - 2
fixed = 322 total (was 324) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
7s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
50s{color} | {color:green} hadoop-yarn-registry in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hadoop-rumen in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
20s{color} | {color:green} hadoop-openstack in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
31s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 42s{color}
| {color:red} hadoop-sls in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
28s{color} | {color:green} hadoop-azure-datalake in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 77m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.sls.TestSLSRunner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HADOOP-13678 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831525/HADOOP-13678.01.patch
|
| Optional Tests |
[jira] [Updated] (HADOOP-13678) Update jackson from 1.9.13 to 2.x in hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-13678: - Release Note: Jackson 1.9.13 dependency was removed from hadoop-tools module. (was: Jackson 1.9.13 dependency was removed from hadoop-openstack module.) > Update jackson from 1.9.13 to 2.x in hadoop-tools > - > > Key: HADOOP-13678 > URL: https://issues.apache.org/jira/browse/HADOOP-13678 > Project: Hadoop Common > Issue Type: Sub-task > Components: tools >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-13678.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13670) Update CHANGES.txt to reflect all the changes in branch-2.7
[
https://issues.apache.org/jira/browse/HADOOP-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545501#comment-15545501
]
Hadoop QA commented on HADOOP-13670:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 777 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
19s{color} | {color:red} The patch 32 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 2m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:c420dfe |
| JIRA Issue | HADOOP-13670 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831543/HADOOP-13670-branch-2.7-003.patch
|
| Optional Tests | asflicense |
| uname | Linux 8ade5126f22e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2.7 / c08346e |
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10655/artifact/patchprocess/whitespace-eol.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10655/artifact/patchprocess/whitespace-tabs.txt
|
| modules | C: hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project hadoop-mapreduce-project U:
. |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10655/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Update CHANGES.txt to reflect all the changes in branch-2.7
> ---
>
> Key: HADOOP-13670
> URL: https://issues.apache.org/jira/browse/HADOOP-13670
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: HADOOP-13670-branch-2.7-003.patch,
> HADOOP-13670-branch-2.7-02.patch, HADOOP-13670-branch-2.7.patch,
> HADOOP-13670.patch
>
>
> When committing to branch-2.7, we need to edit CHANGES.txt. However, there
> are some recent commits to branch-2.7 without editing CHANGES.txt. We need to
> update the change log.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13679) org.apache.hadoop.fs.Path doesn't specify mentioned differences from URIs
[
https://issues.apache.org/jira/browse/HADOOP-13679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Barclay updated HADOOP-13679:
Description:
The documentation comment on {{org.apache.hadoop.fs.Path.Path(String)}} says:
bq. Path strings are URIs, but with unescaped elements and some additional
normalization.
However, neither that comment nor {{org.apache.hadoop.fs.Path}}'s documentation
comment says anything further about exactly what the differences between URI
and {{org.apache.hadoop.fs.Path}} strings are.
(Knowing what {{Path}} strings are supposed to be is important for determining
whether the code is buggy or not.
In particular, it seems that {{Path(String)}} and friends _might_ parse the
scheme portion incorrectly (taking more before a (first) colon rather than that
allowed by the RFC 3986 syntax of:
bq. {{scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )}}
).)
So what exactly are the actual differences between {{Path}} string syntax and
URI syntax?
(I might be able to submit a PR with documentation changes, but I need the raw
information first.)
was:
The documentation comment on {{org.apache.hadoop.fs.Path.Path(String)}} says:
bq. Path strings are URIs, but with unescaped elements and some additional
normalization.
However, neither that comment nor {{org.apache.hadoop.fs.Path}}'s documentation
comment says anything further about exactly what the differences between URI
and {{org.apache.hadoop.fs.Path}} strings are.
(Knowing what {{Path}} strings are supposed to be is important for determining
whether the code is buggy or not.
In particular, it seems that {{Path(String)}} and friends _might_ parse the
scheme portion incorrectly (taking anything before a (first) colon rather than
following the RFC 3986 syntax of:
bq. {{scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )}}
).)
So what exactly are the actual differences between {{Path}} string syntax and
URI syntax?
(I might be able to submit a PR with documentation changes, but I need the raw
information first.)
> org.apache.hadoop.fs.Path doesn't specify mentioned differences from URIs
> -
>
> Key: HADOOP-13679
> URL: https://issues.apache.org/jira/browse/HADOOP-13679
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Reporter: Daniel Barclay
>
> The documentation comment on {{org.apache.hadoop.fs.Path.Path(String)}} says:
> bq. Path strings are URIs, but with unescaped elements and some additional
> normalization.
> However, neither that comment nor {{org.apache.hadoop.fs.Path}}'s
> documentation comment says anything further about exactly what the
> differences between URI and {{org.apache.hadoop.fs.Path}} strings are.
> (Knowing what {{Path}} strings are supposed to be is important for
> determining whether the code is buggy or not.
> In particular, it seems that {{Path(String)}} and friends _might_ parse the
> scheme portion incorrectly (taking more before a (first) colon rather than
> that allowed by the RFC 3986 syntax of:
> bq. {{scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )}}
> ).)
> So what exactly are the actual differences between {{Path}} string syntax and
> URI syntax?
> (I might be able to submit a PR with documentation changes, but I need the
> raw information first.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13670) Update CHANGES.txt to reflect all the changes in branch-2.7
[ https://issues.apache.org/jira/browse/HADOOP-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HADOOP-13670: -- Attachment: HADOOP-13670-branch-2.7-003.patch Uploaded the patch to address all the missing entries..[~ajisakaa] kindly review.. As version rule, I think ,we should not give new features and incompatiable issues with dot and minor releases. > Update CHANGES.txt to reflect all the changes in branch-2.7 > --- > > Key: HADOOP-13670 > URL: https://issues.apache.org/jira/browse/HADOOP-13670 > Project: Hadoop Common > Issue Type: Task >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Blocker > Attachments: HADOOP-13670-branch-2.7-003.patch, > HADOOP-13670-branch-2.7-02.patch, HADOOP-13670-branch-2.7.patch, > HADOOP-13670.patch > > > When committing to branch-2.7, we need to edit CHANGES.txt. However, there > are some recent commits to branch-2.7 without editing CHANGES.txt. We need to > update the change log. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13680) fs.s3a.readahead.range to use getLongBytes
Steve Loughran created HADOOP-13680:
---
Summary: fs.s3a.readahead.range to use getLongBytes
Key: HADOOP-13680
URL: https://issues.apache.org/jira/browse/HADOOP-13680
Project: Hadoop Common
Issue Type: Sub-task
Reporter: Steve Loughran
The {{fs.s3a.readahead.range}} value is measured in bytes, but can be hundreds
of KB. Easier to use getLongBytes and set to things like "300k"
This will be backwards compatible with the existing settings if anyone is using
them, because the no-prefix default will still be bytes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13679) org.apache.hadoop.fs.Path doesn't specify mentioned differences from URIs
[
https://issues.apache.org/jira/browse/HADOOP-13679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Barclay updated HADOOP-13679:
Description:
The documentation comment on {{org.apache.hadoop.fs.Path.Path(String)}} says:
bq. Path strings are URIs, but with unescaped elements and some additional
normalization.
However, neither that comment nor {{org.apache.hadoop.fs.Path}}'s documentation
comment says anything further about exactly what the differences between URI
and {{org.apache.hadoop.fs.Path}} strings are.
(Knowing what {{Path}} strings are supposed to be is important for determining
whether the code is buggy or not.
In particular, it seems that {{Path(String)}} and friends _might_ parse the
scheme portion incorrectly (taking anything before a (first) colon rather than
following the RFC 3986 syntax of:
bq. {{scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )}}
).)
So what exactly are the actual differences between {{Path}} string syntax and
URI syntax?
(I might be able to submit a PR with documentation changes, but I need the raw
information first.)
was:
The documentation comment on {{org.apache.hadoop.fs.Path.Path(String)}} says:
bq. Path strings are URIs, but with unescaped elements and some additional
normalization.
However, neither that comment nor {{org.apache.hadoop.fs.Path}}'s documentation
comment says anything further about exactly what the differences between URI
and {{org.apache.hadoop.fs.Path}} strings are.
(Knowing what {{Path}} strings are supposed to be is important for determining
whether the code is buggy or not.
In particular, {{Path(String)}} and friends seem to parse the scheme portion
incorrectly (taking anything before a (first) colon rather than following the
RFC 3986 syntax of:
bq. {{scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )}}
).)
So what exactly are the actual differences between {{Path}} string syntax and
URI syntax?
(I might be able to submit a PR with documentation changes, but I need the raw
information first.)
> org.apache.hadoop.fs.Path doesn't specify mentioned differences from URIs
> -
>
> Key: HADOOP-13679
> URL: https://issues.apache.org/jira/browse/HADOOP-13679
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Reporter: Daniel Barclay
>
> The documentation comment on {{org.apache.hadoop.fs.Path.Path(String)}} says:
> bq. Path strings are URIs, but with unescaped elements and some additional
> normalization.
> However, neither that comment nor {{org.apache.hadoop.fs.Path}}'s
> documentation comment says anything further about exactly what the
> differences between URI and {{org.apache.hadoop.fs.Path}} strings are.
> (Knowing what {{Path}} strings are supposed to be is important for
> determining whether the code is buggy or not.
> In particular, it seems that {{Path(String)}} and friends _might_ parse the
> scheme portion incorrectly (taking anything before a (first) colon rather
> than following the RFC 3986 syntax of:
> bq. {{scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )}}
> ).)
> So what exactly are the actual differences between {{Path}} string syntax and
> URI syntax?
> (I might be able to submit a PR with documentation changes, but I need the
> raw information first.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13678) Update jackson from 1.9.13 to 2.x in hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13678: --- Hadoop Flags: Incompatible change Release Note: Jackson 1.9.13 dependency was removed from hadoop-openstack module. Status: Patch Available (was: Open) > Update jackson from 1.9.13 to 2.x in hadoop-tools > - > > Key: HADOOP-13678 > URL: https://issues.apache.org/jira/browse/HADOOP-13678 > Project: Hadoop Common > Issue Type: Sub-task > Components: tools >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-13678.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13678) Update jackson from 1.9.13 to 2.x in hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13678: --- Attachment: HADOOP-13678.01.patch > Update jackson from 1.9.13 to 2.x in hadoop-tools > - > > Key: HADOOP-13678 > URL: https://issues.apache.org/jira/browse/HADOOP-13678 > Project: Hadoop Common > Issue Type: Sub-task > Components: tools >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-13678.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13677) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[
https://issues.apache.org/jira/browse/HADOOP-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545332#comment-15545332
]
Hadoop QA commented on HADOOP-13677:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated
0 new + 131 unchanged - 1 fixed = 131 total (was 132) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} hadoop-yarn-server-applicationhistoryservice in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} hadoop-yarn-server-timeline-pluginstorage in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry
generated 0 new + 48 unchanged - 4 fixed = 48 total (was 52) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
25s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
19s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 44s{color}
| {color:red} hadoop-yarn-server-applicationhistoryservice in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
46s{color} | {color:green} hadoop-yarn-server-
[jira] [Created] (HADOOP-13679) org.apache.hadoop.fs.Path doesn't specify mentioned differences from URIs
Daniel Barclay created HADOOP-13679:
---
Summary: org.apache.hadoop.fs.Path doesn't specify mentioned
differences from URIs
Key: HADOOP-13679
URL: https://issues.apache.org/jira/browse/HADOOP-13679
Project: Hadoop Common
Issue Type: Bug
Components: documentation
Reporter: Daniel Barclay
The documentation comment on {{org.apache.hadoop.fs.Path.Path(String)}} says:
bq. Path strings are URIs, but with unescaped elements and some additional
normalization.
However, neither that comment nor {{org.apache.hadoop.fs.Path}}'s documentation
comment says anything further about exactly what the differences between URI
and {{org.apache.hadoop.fs.Path}} strings are.
(Knowing what {{Path}} strings are supposed to be is important for determining
whether the code is buggy or not.
In particular, {{Path(String)}} and friends seem to parse the scheme portion
incorrectly (taking anything before a (first) colon rather than following the
RFC 3986 syntax of:
bq. {{scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )}}
).)
So what exactly are the actual differences between {{Path}} string syntax and
URI syntax?
(I might be able to submit a PR with documentation changes, but I need the raw
information first.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13502) Rename/split fs.contract.is-blobstore flag used by contract tests.
[
https://issues.apache.org/jira/browse/HADOOP-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545288#comment-15545288
]
Chris Nauroth commented on HADOOP-13502:
I'm aware of closed-source implementations that may be using the HCFS tests.
Really this comes down to compatibility policy for the HCFS tests. I don't
think we've discussed that in the past, and there are currently no
audience/stability annotations. If we are committed to backward compatibility,
then we're better off with the approach I took in patch 001. If we are not
committed to backward compatibility for the HCFS tests, then we're free to
change anything we want and repurpose is-blobstore as a purely descriptive
flag. I have a slight preference for maintaining backward compatibility, but
would welcome other opinions.
> Rename/split fs.contract.is-blobstore flag used by contract tests.
> --
>
> Key: HADOOP-13502
> URL: https://issues.apache.org/jira/browse/HADOOP-13502
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Attachments: HADOOP-13502-branch-2.001.patch
>
>
> The {{fs.contract.is-blobstore}} flag guards against execution of several
> contract tests to account for known limitations with blob stores. However,
> the name is not entirely accurate, because it's still possible that a file
> system implemented against a blob store could pass those tests, depending on
> whether or not the implementation matches the semantics of HDFS. This issue
> proposes to rename the flag or split it into different flags with different
> definitions for the semantics covered by the current flag.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13678) Update jackson from 1.9.13 to 2.x in hadoop-tools
Akira Ajisaka created HADOOP-13678: -- Summary: Update jackson from 1.9.13 to 2.x in hadoop-tools Key: HADOOP-13678 URL: https://issues.apache.org/jira/browse/HADOOP-13678 Project: Hadoop Common Issue Type: Sub-task Components: tools Reporter: Akira Ajisaka Assignee: Akira Ajisaka -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13234) Get random port by new ServerSocket(0).getLocalPort() in ServerSocketUtil#getPort
[
https://issues.apache.org/jira/browse/HADOOP-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545234#comment-15545234
]
Hadoop QA commented on HADOOP-13234:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
49s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 23s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 11m 19s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ha.TestZKFailoverController |
| | hadoop.ha.TestZKFailoverControllerStress |
| | hadoop.ha.TestActiveStandbyElectorRealZK |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HADOOP-13234 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831513/HADOOP-13234.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 649c4ea8cbeb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / f61e3d1 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10652/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10652/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10652/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10652/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Get random port by new ServerSo
[jira] [Updated] (HADOOP-13677) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/HADOOP-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13677: --- Attachment: HADOOP-13677.02.patch 02 patch: Fixed javadoc warnings. The test failure is not related to the patch. (YARN-5679) > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: HADOOP-13677 > URL: https://issues.apache.org/jira/browse/HADOOP-13677 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13676) Update jackson from 1.9.13 to 2.x in hadoop-mapreduce
[ https://issues.apache.org/jira/browse/HADOOP-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545200#comment-15545200 ] Akira Ajisaka commented on HADOOP-13676: The test failures are not related to the patch. * TestAMWebApp: MAPREDUCE-6789 * TestMapReduceJobControl: Passed locally > Update jackson from 1.9.13 to 2.x in hadoop-mapreduce > - > > Key: HADOOP-13676 > URL: https://issues.apache.org/jira/browse/HADOOP-13676 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-13676.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13628) Support to retrieve specific property from configuration via REST API
[
https://issues.apache.org/jira/browse/HADOOP-13628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545167#comment-15545167
]
Weiwei Yang commented on HADOOP-13628:
--
Hello [~liuml07]
Majority of the conflicts is from HADOOP-13588, that one add the support of
ACCEPT header in ConfServlet, but it was only committed to trunk. So we have 2
options here,
1. Commit HADOOP-13588 to branch-2 and branch-2.7 as well, then this one.
2. Only commit this one to trunk
I am OK with either option, and a little bit prefer to #2, because this ticket
was initially for the development of YARN-5145 and that one is in trunk. Please
let me know your option. Thank you!
> Support to retrieve specific property from configuration via REST API
> -
>
> Key: HADOOP-13628
> URL: https://issues.apache.org/jira/browse/HADOOP-13628
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
>Affects Versions: 2.7.3
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: 404_error_browser.png, HADOOP-13628.01.patch,
> HADOOP-13628.02.patch, HADOOP-13628.03.patch, HADOOP-13628.04.patch,
> HADOOP-13628.05.patch, HADOOP-13628.06.patch
>
>
> Currently we can use rest API to retrieve all configuration properties per
> daemon, but unable to get a specific property by name. This causes extra
> parse work at client side when dealing with Hadoop configurations, and also
> it's quite over head to send all configuration in a http response over
> network. Propose to support following a {{name}} parameter in the http
> request, by issuing
> {code}
> curl --header "Accept:application/json"
> http://${RM_HOST}/conf?name=yarn.nodemanager.aux-services
> {code}
> get output
> {code}
> {"property"{"key":"yarn.resourcemanager.hostname","value":"${RM_HOST}","isFinal":false,"resource":"yarn-site.xml"}}
> {code}
> This change is fully backwards compatible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13234) Get random port by new ServerSocket(0).getLocalPort() in ServerSocketUtil#getPort
[
https://issues.apache.org/jira/browse/HADOOP-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated HADOOP-13234:
--
Attachment: HADOOP-13234.patch
Uploaded the patch..Kindly review..
Note : As of now, not removing the {{retries}} param.as this method is used by
so many classes.
> Get random port by new ServerSocket(0).getLocalPort() in
> ServerSocketUtil#getPort
> -
>
> Key: HADOOP-13234
> URL: https://issues.apache.org/jira/browse/HADOOP-13234
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: HADOOP-13234.patch
>
>
> As per [~iwasakims] comment from
> [here|https://issues.apache.org/jira/browse/HDFS-10367?focusedCommentId=15275604&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15275604]
> we can get available random port by {{new ServerSocket(0).getLocalPort()}}
> and it's more portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13234) Get random port by new ServerSocket(0).getLocalPort() in ServerSocketUtil#getPort
[
https://issues.apache.org/jira/browse/HADOOP-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated HADOOP-13234:
--
Status: Patch Available (was: Open)
> Get random port by new ServerSocket(0).getLocalPort() in
> ServerSocketUtil#getPort
> -
>
> Key: HADOOP-13234
> URL: https://issues.apache.org/jira/browse/HADOOP-13234
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: HADOOP-13234.patch
>
>
> As per [~iwasakims] comment from
> [here|https://issues.apache.org/jira/browse/HDFS-10367?focusedCommentId=15275604&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15275604]
> we can get available random port by {{new ServerSocket(0).getLocalPort()}}
> and it's more portable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13502) Rename/split fs.contract.is-blobstore flag used by contract tests.
[
https://issues.apache.org/jira/browse/HADOOP-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545033#comment-15545033
]
Steve Loughran commented on HADOOP-13502:
-
What external object stores are we thinking of? We know GCS connector doesn't
have those tests, ADL and WASB are in the code, Allyun is coming in. leaving
only: a fork of swift (which probably doesn't run these tests) and EMR S3.
> Rename/split fs.contract.is-blobstore flag used by contract tests.
> --
>
> Key: HADOOP-13502
> URL: https://issues.apache.org/jira/browse/HADOOP-13502
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Attachments: HADOOP-13502-branch-2.001.patch
>
>
> The {{fs.contract.is-blobstore}} flag guards against execution of several
> contract tests to account for known limitations with blob stores. However,
> the name is not entirely accurate, because it's still possible that a file
> system implemented against a blob store could pass those tests, depending on
> whether or not the implementation matches the semantics of HDFS. This issue
> proposes to rename the flag or split it into different flags with different
> definitions for the semantics covered by the current flag.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13674) S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint.
[ https://issues.apache.org/jira/browse/HADOOP-13674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545025#comment-15545025 ] Steve Loughran commented on HADOOP-13674: - LGTM +1, *provided you get rid of those tabs in the stack trace* > S3A can provide a more detailed error message when accessing a bucket through > an incorrect S3 endpoint. > --- > > Key: HADOOP-13674 > URL: https://issues.apache.org/jira/browse/HADOOP-13674 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Attachments: HADOOP-13674-branch-2.001.patch > > > When accessing the S3 service through a region-specific endpoint, the bucket > must be located in that region. If the client attempts to access a bucket > that is not located in that region, then the service replies with a 301 > redirect and the correct region endpoint. However, the exception thrown by > S3A does not include the correct endpoint. If we included that information > in the exception, it would make it easier for users to diagnose and fix > incorrect configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13672) Extract out jackson calls into an overrideable method in DelegationTokenAuthenticationHandler
[ https://issues.apache.org/jira/browse/HADOOP-13672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545020#comment-15545020 ] Steve Loughran commented on HADOOP-13672: - -1 pending a bit of discussion. # Do you want jackson off the CP entirely? Or just marshall differently but retain Jackson on the CP? As the imports in this class probably mandate having jackson there. # have you tested Solr with this patch and subclass? I'd like to be confident this patch actually meets your needs; I don't want to be going round in circles patching Hadoop until things stabilize > Extract out jackson calls into an overrideable method in > DelegationTokenAuthenticationHandler > - > > Key: HADOOP-13672 > URL: https://issues.apache.org/jira/browse/HADOOP-13672 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Minor > Attachments: HADOOP-13672.patch, HADOOP-13672.patch > > > In Apache Solr, we use hadoop-auth for delegation tokens. However, because of > the following lines, we need to import Jackson (old version). > https://github.com/apache/hadoop/blob/branch-2.7/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java#L279 > If we could extract out the calls to ObjectMapper to another method, so that > at Solr we could override it to do the Map -> json conversion using noggit, > it would be helpful. > Reference: SOLR-9542 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13677) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[
https://issues.apache.org/jira/browse/HADOOP-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15544997#comment-15544997
]
Hadoop QA commented on HADOOP-13677:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
1s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
58s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
44s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated
0 new + 131 unchanged - 1 fixed = 131 total (was 132) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
6s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
13s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry
generated 3 new + 48 unchanged - 4 fixed = 51 total (was 52) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
25s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
17s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 44s{color}
| {color:red} hadoop-yarn-server-applicationhistoryservice in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
45s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
8s{color} | {color:green} hadoop-yarn-server-timeline-pluginstorage in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
44s{color} | {color:green} hadoop-yarn-registry in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 52s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:95
[jira] [Commented] (HADOOP-13676) Update jackson from 1.9.13 to 2.x in hadoop-mapreduce
[
https://issues.apache.org/jira/browse/HADOOP-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15544947#comment-15544947
]
Hadoop QA commented on HADOOP-13676:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} hadoop-mapreduce-project/hadoop-mapreduce-client:
The patch generated 0 new + 413 unchanged - 2 fixed = 413 total (was 415)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
9s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 54s{color}
| {color:red} hadoop-mapreduce-client-app in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}120m 24s{color}
| {color:red} hadoop-mapreduce-client-jobclient in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.mapreduce.v2.app.webapp.TestAMWebApp |
| Timed out junit tests |
org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HADOOP-13676 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831477/HADOOP-13676.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux bc9a324fa513 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| gi
[jira] [Commented] (HADOOP-12977) s3a ignores delete("/", true)
[
https://issues.apache.org/jira/browse/HADOOP-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15544873#comment-15544873
]
Steve Loughran commented on HADOOP-12977:
-
ooh, good point about "//". I'll add a test.
regarding the assert, yes, I'll cut it. I added the second bit in to do a dump
of what's going on, but clearly never hit that codepath again (I think it was
an s3 list inconstency).
> s3a ignores delete("/", true)
> -
>
> Key: HADOOP-12977
> URL: https://issues.apache.org/jira/browse/HADOOP-12977
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Attachments: HADOOP-12977-001.patch, HADOOP-12977-002.patch,
> HADOOP-12977-branch-2-002.patch, HADOOP-12977-branch-2-002.patch
>
>
> if you try to delete the root directory on s3a, you get politely but firmly
> told you can't
> {code}
> 2016-03-30 12:01:44,924 INFO s3a.S3AFileSystem
> (S3AFileSystem.java:delete(638)) - s3a cannot delete the root directory
> {code}
> The semantics of {{rm -rf "/"}} are defined, they are "delete everything
> underneath, while preserving the root dir itself".
> # s3a needs to support this.
> # this skipped through the FS contract tests in
> {{AbstractContractRootDirectoryTest}}; the option of whether deleting / works
> or not should be made configurable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-13677) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/HADOOP-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-13677: -- Assignee: Akira Ajisaka > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: HADOOP-13677 > URL: https://issues.apache.org/jira/browse/HADOOP-13677 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-13677.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13677) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/HADOOP-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13677: --- Hadoop Flags: Incompatible change Release Note: Jackson 1.9.13 dependency was removed from hadoop-yarn-project. Status: Patch Available (was: Open) > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: HADOOP-13677 > URL: https://issues.apache.org/jira/browse/HADOOP-13677 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka > Attachments: HADOOP-13677.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13677) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/HADOOP-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13677: --- Attachment: HADOOP-13677.01.patch > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: HADOOP-13677 > URL: https://issues.apache.org/jira/browse/HADOOP-13677 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka > Attachments: HADOOP-13677.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13677) Update jackson from 1.9.13 to 2.x in hadoop-yarn
Akira Ajisaka created HADOOP-13677: -- Summary: Update jackson from 1.9.13 to 2.x in hadoop-yarn Key: HADOOP-13677 URL: https://issues.apache.org/jira/browse/HADOOP-13677 Project: Hadoop Common Issue Type: Sub-task Reporter: Akira Ajisaka -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13332) Remove jackson 1.9.13 and switch all jackson code to 2.x code line
[ https://issues.apache.org/jira/browse/HADOOP-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13332: --- Status: Open (was: Patch Available) > Remove jackson 1.9.13 and switch all jackson code to 2.x code line > -- > > Key: HADOOP-13332 > URL: https://issues.apache.org/jira/browse/HADOOP-13332 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.8.0 >Reporter: PJ Fanning >Assignee: Akira Ajisaka > Attachments: HADOOP-13332.00.patch, HADOOP-13332.01.patch, > HADOOP-13332.02.patch, HADOOP-13332.03.patch > > > This jackson 1.9 code line is no longer maintained. Upgrade > Most changes from jackson 1.9 to 2.x just involve changing the package name. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13522) Permissions on fs stat added
[ https://issues.apache.org/jira/browse/HADOOP-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13522: --- Status: Open (was: Patch Available) > Permissions on fs stat added > > > Key: HADOOP-13522 > URL: https://issues.apache.org/jira/browse/HADOOP-13522 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.0.0-alpha1 >Reporter: Alex Garbarini >Assignee: Alex Garbarini >Priority: Minor > Attachments: HADOOP-13522.001.patch, HADOOP-13522.002.patch, > HADOOP-13522.003.patch, HADOOP-13522.004.patch, HADOOP-13522.005.patch > > > This patch adds to fs/shell/Stat.java the missing options of %a and %A. > FileStatus already contains the getPermission() method required for returning > symbolic permissions. FsPermission contains the method to return the binary > short, but nothing to present in standard Octal format. > Most UNIX admins base their work on such standard octal permissions. Hence, > this patch also introduces one tiny method to translate the toShort() return > into octal. > Build has already passed unit tests and javadoc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13522) Permissions on fs stat added
[ https://issues.apache.org/jira/browse/HADOOP-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13522: --- Priority: Major (was: Minor) > Permissions on fs stat added > > > Key: HADOOP-13522 > URL: https://issues.apache.org/jira/browse/HADOOP-13522 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.0.0-alpha1 >Reporter: Alex Garbarini >Assignee: Alex Garbarini > Attachments: HADOOP-13522.001.patch, HADOOP-13522.002.patch, > HADOOP-13522.003.patch, HADOOP-13522.004.patch, HADOOP-13522.005.patch > > > This patch adds to fs/shell/Stat.java the missing options of %a and %A. > FileStatus already contains the getPermission() method required for returning > symbolic permissions. FsPermission contains the method to return the binary > short, but nothing to present in standard Octal format. > Most UNIX admins base their work on such standard octal permissions. Hence, > this patch also introduces one tiny method to translate the toShort() return > into octal. > Build has already passed unit tests and javadoc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13522) Permissions on fs stat added
[ https://issues.apache.org/jira/browse/HADOOP-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15544594#comment-15544594 ] Akira Ajisaka commented on HADOOP-13522: Thank you [~algarbar] for updating the patch! Now HADOOP-10892 is committed, so would you rebase the patch for the latest trunk? > Permissions on fs stat added > > > Key: HADOOP-13522 > URL: https://issues.apache.org/jira/browse/HADOOP-13522 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.0.0-alpha1 >Reporter: Alex Garbarini >Assignee: Akira Ajisaka >Priority: Minor > Attachments: HADOOP-13522.001.patch, HADOOP-13522.002.patch, > HADOOP-13522.003.patch, HADOOP-13522.004.patch, HADOOP-13522.005.patch > > > This patch adds to fs/shell/Stat.java the missing options of %a and %A. > FileStatus already contains the getPermission() method required for returning > symbolic permissions. FsPermission contains the method to return the binary > short, but nothing to present in standard Octal format. > Most UNIX admins base their work on such standard octal permissions. Hence, > this patch also introduces one tiny method to translate the toShort() return > into octal. > Build has already passed unit tests and javadoc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13522) Permissions on fs stat added
[ https://issues.apache.org/jira/browse/HADOOP-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15544594#comment-15544594 ] Akira Ajisaka edited comment on HADOOP-13522 at 10/4/16 7:16 AM: - Thank you [~algarbar] for updating the patch! Now HDFS-10892 is committed, so would you rebase the patch for the latest trunk? was (Author: ajisakaa): Thank you [~algarbar] for updating the patch! Now HADOOP-10892 is committed, so would you rebase the patch for the latest trunk? > Permissions on fs stat added > > > Key: HADOOP-13522 > URL: https://issues.apache.org/jira/browse/HADOOP-13522 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.0.0-alpha1 >Reporter: Alex Garbarini >Assignee: Akira Ajisaka >Priority: Minor > Attachments: HADOOP-13522.001.patch, HADOOP-13522.002.patch, > HADOOP-13522.003.patch, HADOOP-13522.004.patch, HADOOP-13522.005.patch > > > This patch adds to fs/shell/Stat.java the missing options of %a and %A. > FileStatus already contains the getPermission() method required for returning > symbolic permissions. FsPermission contains the method to return the binary > short, but nothing to present in standard Octal format. > Most UNIX admins base their work on such standard octal permissions. Hence, > this patch also introduces one tiny method to translate the toShort() return > into octal. > Build has already passed unit tests and javadoc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13522) Permissions on fs stat added
[ https://issues.apache.org/jira/browse/HADOOP-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-13522: --- Assignee: Alex Garbarini (was: Akira Ajisaka) > Permissions on fs stat added > > > Key: HADOOP-13522 > URL: https://issues.apache.org/jira/browse/HADOOP-13522 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.0.0-alpha1 >Reporter: Alex Garbarini >Assignee: Alex Garbarini >Priority: Minor > Attachments: HADOOP-13522.001.patch, HADOOP-13522.002.patch, > HADOOP-13522.003.patch, HADOOP-13522.004.patch, HADOOP-13522.005.patch > > > This patch adds to fs/shell/Stat.java the missing options of %a and %A. > FileStatus already contains the getPermission() method required for returning > symbolic permissions. FsPermission contains the method to return the binary > short, but nothing to present in standard Octal format. > Most UNIX admins base their work on such standard octal permissions. Hence, > this patch also introduces one tiny method to translate the toShort() return > into octal. > Build has already passed unit tests and javadoc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13653) ZKDelegationTokenSecretManager curator client seems to rapidly connect & disconnect from ZK
[
https://issues.apache.org/jira/browse/HADOOP-13653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15544588#comment-15544588
]
Alex Ivanov commented on HADOOP-13653:
--
[~xiaochen], I wonder if the issue we've seen is related to the curator version
used by KMS. I see a very similar exception stack to the one here:
[CURATOR-209|https://issues.apache.org/jira/browse/CURATOR-209]. Do you see any
problem with having KMS use curator 2.10 instead?
> ZKDelegationTokenSecretManager curator client seems to rapidly connect &
> disconnect from ZK
> ---
>
> Key: HADOOP-13653
> URL: https://issues.apache.org/jira/browse/HADOOP-13653
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Reporter: Alex Ivanov
>Priority: Critical
>
> During periods of time, KMS gets in a connect/disconnect loop from Zookeeper.
> It is not clear what causes the connection to be closed. I didn't see any
> issues on the ZK server side, so the issue must reside on client side.
> *Example errors*
> NOTE: I had to filter the logs heavily since they were many GB in size
> (thanks to curator error logging). What is left is an illustration of the
> delegation token creations, and the Zookeeper sessions getting lost and
> re-established over the course of 2 hours.
> {code}
> 2016-09-25 01:43:04,377 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [75027a21ab399aa7789d6907d70fadc4, 46]
> 2016-09-25 01:43:04,557 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [1106d0754d43dcf29324d7be737f51f0, 46]
> 2016-09-25 01:43:11,846 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [4426092c861f49c6ba0c60b49b9539e5, 46]
> 2016-09-25 01:43:48,974 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [a99efff2705d6489deb059098f18818f, 46]
> 2016-09-25 01:43:49,174 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [398b5962fd647880961ba5e86a77b414, 46]
> 2016-09-25 01:44:03,359 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [413187e62a21b5459422b5c524315d06, 46]
> 2016-09-25 01:44:03,625 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [7cc2c0d82edd40e7e6f6f40af20d04d3, 46]
> 2016-09-25 01:44:06,062 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [bd9394fce20607c12bc00104bea49284, 46]
> 2016-09-25 01:44:07,134 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [7dad3bd10526517e5e1cfccd2e96074a, 46]
> 2016-09-25 01:44:07,230 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [a712ed40687580647d070c9c7f525e15, 46]
> 2016-09-25 01:44:48,481 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [44bfefa31192c68e3cc053eec4e57e14, 46]
> 2016-09-25 01:44:48,522 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [67efc2aa65eeba701ad7d3d7bab51def, 46]
> 2016-09-25 01:44:50,259 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [b43e641f58dfbd2c72550ab6804f37d1, 46]
> 2016-09-25 01:44:54,271 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [ac2fbcf404c633759b75e6d6aae00e05, 46]
> 2016-09-25 01:44:56,141 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [cdbd224079a4a10400d00d0b8eece008, 46]
> 2016-09-25 01:45:01,328 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [e03218f4835524f3d05519d27bb04e35, 46]
> 2016-09-25 01:45:02,728 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [569ae6d666d584b6843fffc47a63d147, 46]
> 2016-09-25 01:45:02,832 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [c9048271483da234c12f75569b9513c6, 46]
> 2016-09-25 01:45:05,536 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [f519d621389e41b63e8d92b4cb15f832, 46]
> 2016-09-25 01:45:07,886 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [45cf6ba58b2bb348ac5e88fa18fe9dad, 46]
> 2016-09-25 01:47:24,346 WARN ConnectionState - Connection attempt
> unsuccessful after 66294 (greater than max timeout of 6). Resetting
> connection and trying again with a new connection.
> 2016-09-25 01:47:25,120 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [f160a865db69ef33548f146c9b3b84c6, 46]
> 2016-09-25 01:47:25,276 INFO AbstractDelegationTokenSecretManager - Creating
> password for identifier: [9d60add471464e01ef691c43bd901d96, 46]
> 2016-09-25 01:47:28,739 INFO Abstr
