[jira] [Updated] (HADOOP-16712) Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xieming Li updated HADOOP-16712: Attachment: HADOOP-16712.001.patch Status: Patch Available (was: Open) > Config ha.failover-controller.active-standby-elector.zk.op.retries is not in > core-default.xml > - > > Key: HADOOP-16712 > URL: https://issues.apache.org/jira/browse/HADOOP-16712 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Xieming Li >Priority: Trivial > Attachments: HADOOP-16712.001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15097) AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path
[
https://issues.apache.org/jira/browse/HADOOP-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974801#comment-16974801
]
Hudson commented on HADOOP-15097:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17643 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17643/])
HADOOP-15097. AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive
(aajisaka: rev 92c28c100ee1aa414948cd510321ad13cb8639bc)
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractDeleteTest.java
> AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading
> path
> ---
>
> Key: HADOOP-15097
> URL: https://issues.apache.org/jira/browse/HADOOP-15097
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0-beta1
>Reporter: zhoutai.zt
>Assignee: Xieming Li
>Priority: Minor
> Labels: newbie
> Fix For: 3.3.0, 3.1.4, 3.2.2, 2.10.1, 2.11.0
>
> Attachments: HADOOP-15097.001.patch
>
>
> @Test
> public void testDeleteNonEmptyDirRecursive() throws Throwable {
> Path path = path("{color:red}testDeleteNonEmptyDirNonRecursive{color}");
> mkdirs(path);
> Path file = new Path(path, "childfile");
> ContractTestUtils.writeTextFile(getFileSystem(), file, "goodbye, world",
> true);
> assertDeleted(path, true);
> assertPathDoesNotExist("not deleted", file);
> }
> change testDeleteNonEmptyDirNonRecursive to testDeleteNonEmptyDirRecursive
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15097) AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path
[
https://issues.apache.org/jira/browse/HADOOP-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-15097:
---
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed this to trunk, branch-3.2, branch-3.1, branch-2, and branch-2.10.
Thanks [~risyomei] for the contribution and thanks [~zhoutai.zt] for the report.
> AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading
> path
> ---
>
> Key: HADOOP-15097
> URL: https://issues.apache.org/jira/browse/HADOOP-15097
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0-beta1
>Reporter: zhoutai.zt
>Assignee: Xieming Li
>Priority: Minor
> Labels: newbie
> Fix For: 3.3.0, 3.1.4, 3.2.2, 2.10.1, 2.11.0
>
> Attachments: HADOOP-15097.001.patch
>
>
> @Test
> public void testDeleteNonEmptyDirRecursive() throws Throwable {
> Path path = path("{color:red}testDeleteNonEmptyDirNonRecursive{color}");
> mkdirs(path);
> Path file = new Path(path, "childfile");
> ContractTestUtils.writeTextFile(getFileSystem(), file, "goodbye, world",
> true);
> assertDeleted(path, true);
> assertPathDoesNotExist("not deleted", file);
> }
> change testDeleteNonEmptyDirNonRecursive to testDeleteNonEmptyDirRecursive
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15097) AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path
[
https://issues.apache.org/jira/browse/HADOOP-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-15097:
---
Fix Version/s: 2.11.0
2.10.1
3.2.2
3.1.4
3.3.0
> AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading
> path
> ---
>
> Key: HADOOP-15097
> URL: https://issues.apache.org/jira/browse/HADOOP-15097
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0-beta1
>Reporter: zhoutai.zt
>Assignee: Xieming Li
>Priority: Minor
> Labels: newbie
> Fix For: 3.3.0, 3.1.4, 3.2.2, 2.10.1, 2.11.0
>
> Attachments: HADOOP-15097.001.patch
>
>
> @Test
> public void testDeleteNonEmptyDirRecursive() throws Throwable {
> Path path = path("{color:red}testDeleteNonEmptyDirNonRecursive{color}");
> mkdirs(path);
> Path file = new Path(path, "childfile");
> ContractTestUtils.writeTextFile(getFileSystem(), file, "goodbye, world",
> true);
> assertDeleted(path, true);
> assertPathDoesNotExist("not deleted", file);
> }
> change testDeleteNonEmptyDirNonRecursive to testDeleteNonEmptyDirRecursive
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15097) AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path
[
https://issues.apache.org/jira/browse/HADOOP-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974790#comment-16974790
]
Akira Ajisaka commented on HADOOP-15097:
+1, committing this.
> AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading
> path
> ---
>
> Key: HADOOP-15097
> URL: https://issues.apache.org/jira/browse/HADOOP-15097
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.0.0-beta1
>Reporter: zhoutai.zt
>Assignee: Xieming Li
>Priority: Minor
> Labels: newbie
> Attachments: HADOOP-15097.001.patch
>
>
> @Test
> public void testDeleteNonEmptyDirRecursive() throws Throwable {
> Path path = path("{color:red}testDeleteNonEmptyDirNonRecursive{color}");
> mkdirs(path);
> Path file = new Path(path, "childfile");
> ContractTestUtils.writeTextFile(getFileSystem(), file, "goodbye, world",
> true);
> assertDeleted(path, true);
> assertPathDoesNotExist("not deleted", file);
> }
> change testDeleteNonEmptyDirNonRecursive to testDeleteNonEmptyDirRecursive
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-16712) Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xieming Li reassigned HADOOP-16712: --- Assignee: Xieming Li > Config ha.failover-controller.active-standby-elector.zk.op.retries is not in > core-default.xml > - > > Key: HADOOP-16712 > URL: https://issues.apache.org/jira/browse/HADOOP-16712 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Xieming Li >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16548) ABFS: Config to enable/disable flush operation
[ https://issues.apache.org/jira/browse/HADOOP-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974613#comment-16974613 ] Mandar Inamdar commented on HADOOP-16548: - Hi [~snvijaya], [~DanielZhou], [~ste...@apache.org], would be interested in having this patch committed to branch-3.2. In one of the use cases am benefiting with the similar change of no-op for Flush. I would like to check if you would like to have a PR for this or if the committers would be able to cherry pick it ? Thanks! > ABFS: Config to enable/disable flush operation > -- > > Key: HADOOP-16548 > URL: https://issues.apache.org/jira/browse/HADOOP-16548 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Bilahari T H >Assignee: Sneha Vijayarajan >Priority: Minor > Fix For: 3.3.0 > > > Make flush operation enabled/disabled through configuration. This is part of > performance improvements for ABFS driver. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jeeteshm commented on issue #1611: HADOOP-16612 Track Azure Blob File System client-perceived latency
jeeteshm commented on issue #1611: HADOOP-16612 Track Azure Blob File System client-perceived latency URL: https://github.com/apache/hadoop/pull/1611#issuecomment-554084244 Thank you @steveloughran for your attention! I have fixed all the javadoc warnings arising out of this PR. The test results are mentioned in the JIRA here: https://issues.apache.org/jira/browse/HADOOP-16612 Patch URL: https://issues.apache.org/jira/secure/attachment/12985886/HADOOP-16612.004.patch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16612) Track Azure Blob File System client-perceived latency
[ https://issues.apache.org/jira/browse/HADOOP-16612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974609#comment-16974609 ] Jeetesh Mangwani commented on HADOOP-16612: --- Here are the test results for the patch https://issues.apache.org/jira/secure/attachment/12985323/HADOOP-16612.004.patch: -- non-xns: [INFO] Tests run: 52, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 395, Failures: 0, Errors: 0, Skipped: 207 [WARNING] Tests run: 192, Failures: 0, Errors: 0, Skipped: 24 xns: [INFO] Tests run: 52, Failures: 0, Errors: 0, Skipped: 0 [ERROR] Failures: [ERROR] ITestGetNameSpaceEnabled.testXNSAccount:51->Assert.assertTrue:41->Assert.fail:88 Expecting getIsNamespaceEnabled() return true [INFO] [ERROR] Tests run: 395, Failures: 1, Errors: 0, Skipped: 207 [WARNING] Tests run: 192, Failures: 0, Errors: 0, Skipped: 24 - Comments: 1. ITestGetNameSpaceEnabled.testNonXNSAccount: fails because the HTTP response status is not 400, but is 404 > Track Azure Blob File System client-perceived latency > - > > Key: HADOOP-16612 > URL: https://issues.apache.org/jira/browse/HADOOP-16612 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, hdfs-client >Reporter: Jeetesh Mangwani >Assignee: Jeetesh Mangwani >Priority: Major > Attachments: HADOOP-16612.001.patch, HADOOP-16612.002.patch, > HADOOP-16612.003.patch, HADOOP-16612.004.patch > > > Track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency > in the Hadoop ABFS driver. > The latency information is sent back to the ADLS Gen 2 REST API endpoints in > the subsequent requests. > Here's the PR: https://github.com/apache/hadoop/pull/1611 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16612) Track Azure Blob File System client-perceived latency
[
https://issues.apache.org/jira/browse/HADOOP-16612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974601#comment-16974601
]
Hadoop QA commented on HADOOP-16612:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 40s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 43s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
17s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:104ccca9169 |
| JIRA Issue | HADOOP-16612 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12985886/HADOOP-16612.004.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 2920f9448104 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1
05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d0302d3 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16670/testReport/ |
| Max. process+thread count | 309 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16670/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Track Azure Blob File
[jira] [Updated] (HADOOP-16612) Track Azure Blob File System client-perceived latency
[ https://issues.apache.org/jira/browse/HADOOP-16612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeetesh Mangwani updated HADOOP-16612: -- Attachment: HADOOP-16612.004.patch > Track Azure Blob File System client-perceived latency > - > > Key: HADOOP-16612 > URL: https://issues.apache.org/jira/browse/HADOOP-16612 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, hdfs-client >Reporter: Jeetesh Mangwani >Assignee: Jeetesh Mangwani >Priority: Major > Attachments: HADOOP-16612.001.patch, HADOOP-16612.002.patch, > HADOOP-16612.003.patch, HADOOP-16612.004.patch > > > Track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency > in the Hadoop ABFS driver. > The latency information is sent back to the ADLS Gen 2 REST API endpoints in > the subsequent requests. > Here's the PR: https://github.com/apache/hadoop/pull/1611 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16712) Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml
Wei-Chiu Chuang created HADOOP-16712: Summary: Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml Key: HADOOP-16712 URL: https://issues.apache.org/jira/browse/HADOOP-16712 Project: Hadoop Common Issue Type: Improvement Reporter: Wei-Chiu Chuang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16711) With S3Guard + authmode, consider skipping "verifyBuckets" check in S3A fs init()
[ https://issues.apache.org/jira/browse/HADOOP-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-16711: -- Attachment: HADOOP-16711.prelim.1.patch > With S3Guard + authmode, consider skipping "verifyBuckets" check in S3A fs > init() > - > > Key: HADOOP-16711 > URL: https://issues.apache.org/jira/browse/HADOOP-16711 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Rajesh Balamohan >Priority: Minor > Labels: performance > Attachments: HADOOP-16711.prelim.1.patch > > > When authoritative mode is enabled with s3guard, it would be good to skip > verifyBuckets call during S3A filesystem init(). This would save call to S3 > during init method. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16711) With S3Guard + authmode, consider skipping "verifyBuckets" check in S3A fs init()
Rajesh Balamohan created HADOOP-16711: - Summary: With S3Guard + authmode, consider skipping "verifyBuckets" check in S3A fs init() Key: HADOOP-16711 URL: https://issues.apache.org/jira/browse/HADOOP-16711 Project: Hadoop Common Issue Type: Sub-task Components: fs/s3 Reporter: Rajesh Balamohan When authoritative mode is enabled with s3guard, it would be good to skip verifyBuckets call during S3A filesystem init(). This would save call to S3 during init method. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16691) Unify Logging in UserGroupInformation
[
https://issues.apache.org/jira/browse/HADOOP-16691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974526#comment-16974526
]
Hudson commented on HADOOP-16691:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17642 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17642/])
HADOOP-16691. Unify Logging in UserGroupInformation. (elgoiri: rev
d0302d3849cc4f801f3b7d417b2c7bd7378d51e9)
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
> Unify Logging in UserGroupInformation
> -
>
> Key: HADOOP-16691
> URL: https://issues.apache.org/jira/browse/HADOOP-16691
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-16691.1.patch
>
>
> Unify logging style in {{UserGroupInformation}} class.
> # Remove superfluous logging guards
> # Remove superfluous String.format
> # Simplify
> # Other
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16691) Unify Logging in UserGroupInformation
[
https://issues.apache.org/jira/browse/HADOOP-16691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974520#comment-16974520
]
Íñigo Goiri commented on HADOOP-16691:
--
Thanks [~belugabehr] for the patch.
Merged the PR.
For the next one use just PR or patch.
> Unify Logging in UserGroupInformation
> -
>
> Key: HADOOP-16691
> URL: https://issues.apache.org/jira/browse/HADOOP-16691
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-16691.1.patch
>
>
> Unify logging style in {{UserGroupInformation}} class.
> # Remove superfluous logging guards
> # Remove superfluous String.format
> # Simplify
> # Other
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16691) Unify Logging in UserGroupInformation
[
https://issues.apache.org/jira/browse/HADOOP-16691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Íñigo Goiri updated HADOOP-16691:
-
Fix Version/s: 3.3.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Unify Logging in UserGroupInformation
> -
>
> Key: HADOOP-16691
> URL: https://issues.apache.org/jira/browse/HADOOP-16691
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-16691.1.patch
>
>
> Unify logging style in {{UserGroupInformation}} class.
> # Remove superfluous logging guards
> # Remove superfluous String.format
> # Simplify
> # Other
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri merged pull request #1716: HADOOP-16691: Unify Logging in UserGroupInformation
goiri merged pull request #1716: HADOOP-16691: Unify Logging in UserGroupInformation URL: https://github.com/apache/hadoop/pull/1716 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16705) MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log
[ https://issues.apache.org/jira/browse/HADOOP-16705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974513#comment-16974513 ] Hudson commented on HADOOP-16705: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17641 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17641/]) HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the (elgoiri: rev c73334a924d2009d136a8ee62278aaac53418a24) * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MBeanInfoBuilder.java > MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug > log > > > Key: HADOOP-16705 > URL: https://issues.apache.org/jira/browse/HADOOP-16705 > Project: Hadoop Common > Issue Type: Improvement > Components: metrics >Affects Versions: 2.9.2 >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.3.0 > > Attachments: mbeaninfobuilder.JPG > > > MBeanInfoBuilder's get() method DEBUG logs all the MBeanAttributeInfo > attributes that it gathered. This can have a high memory churn that can be > easily avoided. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16708) HadoopExecutors cleanup to only log at debug
[
https://issues.apache.org/jira/browse/HADOOP-16708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974501#comment-16974501
]
Hudson commented on HADOOP-16708:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17640 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17640/])
HADOOP-16708. HadoopExecutors cleanup to only log at debug. (stevel: rev
a139a8761054906accadf9ebbabcb16528325991)
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/concurrent/HadoopExecutors.java
> HadoopExecutors cleanup to only log at debug
>
>
> Key: HADOOP-16708
> URL: https://issues.apache.org/jira/browse/HADOOP-16708
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: David Mollitor
>Priority: Minor
>
> HadoopExecutors cleanup to only log at debug
> HADOOP-16181 means that executor shutdown now logs @ info
> this means every single hadoop fs command against s3 has four lines of
> information which is entirely irrelevant to all users.
> {code}
> 2019-11-13 17:32:58,654 [shutdown-hook-0] DEBUG s3a.S3AFileSystem
> (S3AFileSystem.java:close(3126)) - Filesystem s3a://hwdev-steve-ireland-new
> is closed
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(118)) - Gracefully shutting down executor
> service. Waiting max 30 SECONDS
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(129)) - Succesfully shutdown executor service
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(118)) - Gracefully shutting down executor
> service. Waiting max 30 SECONDS
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(129)) - Succesfully shutdown executor service
> {code}
> We don't need to inflict this on anyone. Proposed: downgrade the logs to
> info. No need for tests
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16705) MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log
[ https://issues.apache.org/jira/browse/HADOOP-16705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HADOOP-16705: - Fix Version/s: 3.3.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug > log > > > Key: HADOOP-16705 > URL: https://issues.apache.org/jira/browse/HADOOP-16705 > Project: Hadoop Common > Issue Type: Improvement > Components: metrics >Affects Versions: 2.9.2 >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.3.0 > > Attachments: mbeaninfobuilder.JPG > > > MBeanInfoBuilder's get() method DEBUG logs all the MBeanAttributeInfo > attributes that it gathered. This can have a high memory churn that can be > easily avoided. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri merged pull request #1715: HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log
goiri merged pull request #1715: HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log URL: https://github.com/apache/hadoop/pull/1715 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16705) MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log
[ https://issues.apache.org/jira/browse/HADOOP-16705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974495#comment-16974495 ] Íñigo Goiri commented on HADOOP-16705: -- Yetus showed in the PR. Merged. Thanks [~lukmajercak] for the fix. > MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug > log > > > Key: HADOOP-16705 > URL: https://issues.apache.org/jira/browse/HADOOP-16705 > Project: Hadoop Common > Issue Type: Improvement > Components: metrics >Affects Versions: 2.9.2 >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.3.0 > > Attachments: mbeaninfobuilder.JPG > > > MBeanInfoBuilder's get() method DEBUG logs all the MBeanAttributeInfo > attributes that it gathered. This can have a high memory churn that can be > easily avoided. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16708) HadoopExecutors cleanup to only log at debug
[
https://issues.apache.org/jira/browse/HADOOP-16708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974493#comment-16974493
]
Steve Loughran commented on HADOOP-16708:
-
+1 - committed!
> HadoopExecutors cleanup to only log at debug
>
>
> Key: HADOOP-16708
> URL: https://issues.apache.org/jira/browse/HADOOP-16708
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: David Mollitor
>Priority: Minor
>
> HadoopExecutors cleanup to only log at debug
> HADOOP-16181 means that executor shutdown now logs @ info
> this means every single hadoop fs command against s3 has four lines of
> information which is entirely irrelevant to all users.
> {code}
> 2019-11-13 17:32:58,654 [shutdown-hook-0] DEBUG s3a.S3AFileSystem
> (S3AFileSystem.java:close(3126)) - Filesystem s3a://hwdev-steve-ireland-new
> is closed
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(118)) - Gracefully shutting down executor
> service. Waiting max 30 SECONDS
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(129)) - Succesfully shutdown executor service
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(118)) - Gracefully shutting down executor
> service. Waiting max 30 SECONDS
> 2019-11-13 17:32:58,656 [shutdown-hook-0] INFO s3a.S3AFileSystem
> (HadoopExecutors.java:shutdown(129)) - Succesfully shutdown executor service
> {code}
> We don't need to inflict this on anyone. Proposed: downgrade the logs to
> info. No need for tests
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1717: HADOOP-16708: HadoopExecutors cleanup to only log at debug
steveloughran commented on issue #1717: HADOOP-16708: HadoopExecutors cleanup to only log at debug URL: https://github.com/apache/hadoop/pull/1717#issuecomment-554002717 +1, merged in This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran closed pull request #1717: HADOOP-16708: HadoopExecutors cleanup to only log at debug
steveloughran closed pull request #1717: HADOOP-16708: HadoopExecutors cleanup to only log at debug URL: https://github.com/apache/hadoop/pull/1717 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on issue #1695: HADOOP-16685: FileSystem#listStatusIterator does not check if given path exists
jojochuang commented on issue #1695: HADOOP-16685: FileSystem#listStatusIterator does not check if given path exists URL: https://github.com/apache/hadoop/pull/1695#issuecomment-553994169 The patch itself is good to me. The above discussion aren't related to the patch. Not sure about cloud connectors, but every time I see a potential area in HDFS that worths optimizing and I don't, it always comes back biting me :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] sahilTakiar commented on issue #1695: HADOOP-16685: FileSystem#listStatusIterator does not check if given path exists
sahilTakiar commented on issue #1695: HADOOP-16685: FileSystem#listStatusIterator does not check if given path exists URL: https://github.com/apache/hadoop/pull/1695#issuecomment-553989439 Sounds good. So does this patch look okay? Any other comments before it can be merged? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16683) Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException
[
https://issues.apache.org/jira/browse/HADOOP-16683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974439#comment-16974439
]
Hadoop QA commented on HADOOP-16683:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 40m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} branch-3.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
38s{color} | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
15s{color} | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 58s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} branch-3.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 51s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
41s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}142m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.4 Server=19.03.4 Image:yetus/hadoop:080e9d0f9b3 |
| JIRA Issue | HADOOP-16683 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12985863/HADOOP-16683.branch-3.1.001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 1d3e51e35c77 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1
05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-3.1 / 5cf36aa |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16668/testReport/ |
| Max. process+thread count | 1358 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16668/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped
> AccessControlException
>
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1711: HADOOP-16455. ABFS: Implement FileSystem.access() method.
hadoop-yetus removed a comment on issue #1711: HADOOP-16455. ABFS: Implement FileSystem.access() method. URL: https://github.com/apache/hadoop/pull/1711#issuecomment-553270481 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1092 | trunk passed | | +1 | compile | 32 | trunk passed | | +1 | checkstyle | 25 | trunk passed | | +1 | mvnsite | 35 | trunk passed | | +1 | shadedclient | 796 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 28 | trunk passed | | 0 | spotbugs | 52 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 49 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 28 | the patch passed | | +1 | compile | 25 | the patch passed | | +1 | javac | 25 | the patch passed | | -0 | checkstyle | 18 | hadoop-tools/hadoop-azure: The patch generated 3 new + 6 unchanged - 1 fixed = 9 total (was 7) | | +1 | mvnsite | 27 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 801 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 24 | the patch passed | | -1 | findbugs | 57 | hadoop-tools/hadoop-azure generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 83 | hadoop-azure in the patch passed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 3305 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-tools/hadoop-azure | | | Dead store to e in org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.throwFileNotFoundIfNotExists(String) At AzureBlobFileSystemStore.java:org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.throwFileNotFoundIfNotExists(String) At AzureBlobFileSystemStore.java:[line 288] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1711 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux cf7a5a1e0614 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / df6b316 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/2/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/2/artifact/out/new-findbugs-hadoop-tools_hadoop-azure.html | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/2/testReport/ | | Max. process+thread count | 411 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1711: HADOOP-16455. ABFS: Implement FileSystem.access() method.
hadoop-yetus removed a comment on issue #1711: HADOOP-16455. ABFS: Implement FileSystem.access() method. URL: https://github.com/apache/hadoop/pull/1711#issuecomment-553241843 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 1777 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1087 | trunk passed | | +1 | compile | 31 | trunk passed | | +1 | checkstyle | 26 | trunk passed | | +1 | mvnsite | 36 | trunk passed | | +1 | shadedclient | 808 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 28 | trunk passed | | 0 | spotbugs | 52 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 51 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 28 | the patch passed | | +1 | compile | 24 | the patch passed | | +1 | javac | 24 | the patch passed | | -0 | checkstyle | 18 | hadoop-tools/hadoop-azure: The patch generated 3 new + 6 unchanged - 1 fixed = 9 total (was 7) | | +1 | mvnsite | 28 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 805 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 24 | the patch passed | | +1 | findbugs | 54 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 82 | hadoop-azure in the patch passed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 5047 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1711 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 3d8a165b0545 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / df6b316 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/1/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/1/testReport/ | | Max. process+thread count | 453 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1611: HADOOP-16612 Track Azure Blob File System client-perceived latency
steveloughran commented on issue #1611: HADOOP-16612 Track Azure Blob File
System client-perceived latency
URL: https://github.com/apache/hadoop/pull/1611#issuecomment-553958151
Da Zhou seems away right now, I'll have a go at merging. However before I Do
that, I have just run a mvn javadoc:javadoc and it found a new error
```
[WARNING] ^
[WARNING]
/Users/stevel/Hadoop/commit/apache-hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java:481:
warning: no @return
[WARNING] public boolean shouldTrackLatency() {
[WARNING] ^
```
Can you fix that?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16710) testing_azure.md documentation is misleading
[
https://issues.apache.org/jira/browse/HADOOP-16710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974380#comment-16974380
]
Hadoop QA commented on HADOOP-16710:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
33m 2s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 47s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.4 Server=19.03.4 Image:yetus/hadoop:104ccca9169 |
| JIRA Issue | HADOOP-16710 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12985866/HADOOP-16710.001.patch
|
| Optional Tests | dupname asflicense mvnsite |
| uname | Linux 834465bce4da 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1
05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 73a386a |
| maven | version: Apache Maven 3.3.9 |
| Max. process+thread count | 319 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16669/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> testing_azure.md documentation is misleading
>
>
> Key: HADOOP-16710
> URL: https://issues.apache.org/jira/browse/HADOOP-16710
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, test
>Affects Versions: 3.2.0
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Major
> Attachments: HADOOP-16710.001.patch
>
>
> testing_azure.md states that "-Dparallel-tests" will run all the integration
> tests in parallel.
> But in fact using -Dparallel-tests without any value actually skips the
> integration tests and runs only the unit tests.
> The reason is that to activate a profile which is able to run ITs in parallel
> requires parallel-tests property to have a value (abfs, wasb or 'both').
> sequential-tests profile defines !parallel-tests as value which means that
> the property should not even be mentioned.
> Please check the output of help:active-profiles command:
>
> {code:java}
> cd hadoop-tools/hadoop-azure
> andrasbokor$ mvn help:active-profiles -Dparallel-tests=abfs
> - parallel-tests-abfs (source: org.apache.hadoop:hadoop-azure:3.3.0-SNAPSHOT)
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT) {code}
> {code:java}
> andrasbokor$ mvn help:active-profiles -Dparallel-tests
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> {code}
> {code:java}
> mvn help:active-profiles
> - sequential-tests (source: org.apache.hadoop:hadoop-azure:3.3.0-SNAPSHOT)
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT){code}
> The help:active-profiles shows that
[jira] [Updated] (HADOOP-16710) testing_azure.md documentation is misleading
[
https://issues.apache.org/jira/browse/HADOOP-16710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor updated HADOOP-16710:
--
Component/s: test
> testing_azure.md documentation is misleading
>
>
> Key: HADOOP-16710
> URL: https://issues.apache.org/jira/browse/HADOOP-16710
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, test
>Affects Versions: 3.2.0
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Major
> Attachments: HADOOP-16710.001.patch
>
>
> testing_azure.md states that "-Dparallel-tests" will run all the integration
> tests in parallel.
> But in fact using -Dparallel-tests without any value actually skips the
> integration tests and runs only the unit tests.
> The reason is that to activate a profile which is able to run ITs in parallel
> requires parallel-tests property to have a value (abfs, wasb or 'both').
> sequential-tests profile defines !parallel-tests as value which means that
> the property should not even be mentioned.
> Please check the output of help:active-profiles command:
>
> {code:java}
> cd hadoop-tools/hadoop-azure
> andrasbokor$ mvn help:active-profiles -Dparallel-tests=abfs
> - parallel-tests-abfs (source: org.apache.hadoop:hadoop-azure:3.3.0-SNAPSHOT)
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT) {code}
> {code:java}
> andrasbokor$ mvn help:active-profiles -Dparallel-tests
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> {code}
> {code:java}
> mvn help:active-profiles
> - sequential-tests (source: org.apache.hadoop:hadoop-azure:3.3.0-SNAPSHOT)
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT){code}
> The help:active-profiles shows that -Dparallel-tests does not add any IT
> related profiles and results in skipping all the integration tests during
> verify phrase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16710) testing_azure.md documentation is misleading
[
https://issues.apache.org/jira/browse/HADOOP-16710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor updated HADOOP-16710:
--
Status: Patch Available (was: Open)
> testing_azure.md documentation is misleading
>
>
> Key: HADOOP-16710
> URL: https://issues.apache.org/jira/browse/HADOOP-16710
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 3.2.0
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Major
> Attachments: HADOOP-16710.001.patch
>
>
> testing_azure.md states that "-Dparallel-tests" will run all the integration
> tests in parallel.
> But in fact using -Dparallel-tests without any value actually skips the
> integration tests and runs only the unit tests.
> The reason is that to activate a profile which is able to run ITs in parallel
> requires parallel-tests property to have a value (abfs, wasb or 'both').
> sequential-tests profile defines !parallel-tests as value which means that
> the property should not even be mentioned.
> Please check the output of help:active-profiles command:
>
> {code:java}
> cd hadoop-tools/hadoop-azure
> andrasbokor$ mvn help:active-profiles -Dparallel-tests=abfs
> - parallel-tests-abfs (source: org.apache.hadoop:hadoop-azure:3.3.0-SNAPSHOT)
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT) {code}
> {code:java}
> andrasbokor$ mvn help:active-profiles -Dparallel-tests
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> {code}
> {code:java}
> mvn help:active-profiles
> - sequential-tests (source: org.apache.hadoop:hadoop-azure:3.3.0-SNAPSHOT)
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT){code}
> The help:active-profiles shows that -Dparallel-tests does not add any IT
> related profiles and results in skipping all the integration tests during
> verify phrase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16710) testing_azure.md documentation is misleading
[
https://issues.apache.org/jira/browse/HADOOP-16710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor updated HADOOP-16710:
--
Attachment: HADOOP-16710.001.patch
> testing_azure.md documentation is misleading
>
>
> Key: HADOOP-16710
> URL: https://issues.apache.org/jira/browse/HADOOP-16710
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 3.2.0
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Major
> Attachments: HADOOP-16710.001.patch
>
>
> testing_azure.md states that "-Dparallel-tests" will run all the integration
> tests in parallel.
> But in fact using -Dparallel-tests without any value actually skips the
> integration tests and runs only the unit tests.
> The reason is that to activate a profile which is able to run ITs in parallel
> requires parallel-tests property to have a value (abfs, wasb or 'both').
> sequential-tests profile defines !parallel-tests as value which means that
> the property should not even be mentioned.
> Please check the output of help:active-profiles command:
>
> {code:java}
> cd hadoop-tools/hadoop-azure
> andrasbokor$ mvn help:active-profiles -Dparallel-tests=abfs
> - parallel-tests-abfs (source: org.apache.hadoop:hadoop-azure:3.3.0-SNAPSHOT)
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT) {code}
> {code:java}
> andrasbokor$ mvn help:active-profiles -Dparallel-tests
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> {code}
> {code:java}
> mvn help:active-profiles
> - sequential-tests (source: org.apache.hadoop:hadoop-azure:3.3.0-SNAPSHOT)
> - os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
> - hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT){code}
> The help:active-profiles shows that -Dparallel-tests does not add any IT
> related profiles and results in skipping all the integration tests during
> verify phrase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16710) testing_azure.md documentation is misleading
Andras Bokor created HADOOP-16710:
-
Summary: testing_azure.md documentation is misleading
Key: HADOOP-16710
URL: https://issues.apache.org/jira/browse/HADOOP-16710
Project: Hadoop Common
Issue Type: Bug
Components: fs/azure
Affects Versions: 3.2.0
Reporter: Andras Bokor
Assignee: Andras Bokor
testing_azure.md states that "-Dparallel-tests" will run all the integration
tests in parallel.
But in fact using -Dparallel-tests without any value actually skips the
integration tests and runs only the unit tests.
The reason is that to activate a profile which is able to run ITs in parallel
requires parallel-tests property to have a value (abfs, wasb or 'both').
sequential-tests profile defines !parallel-tests as value which means that the
property should not even be mentioned.
Please check the output of help:active-profiles command:
{code:java}
cd hadoop-tools/hadoop-azure
andrasbokor$ mvn help:active-profiles -Dparallel-tests=abfs
- parallel-tests-abfs (source: org.apache.hadoop:hadoop-azure:3.3.0-SNAPSHOT)
- os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
- hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT) {code}
{code:java}
andrasbokor$ mvn help:active-profiles -Dparallel-tests
- os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
- hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
{code}
{code:java}
mvn help:active-profiles
- sequential-tests (source: org.apache.hadoop:hadoop-azure:3.3.0-SNAPSHOT)
- os.mac (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT)
- hbase1 (source: org.apache.hadoop:hadoop-project:3.3.0-SNAPSHOT){code}
The help:active-profiles shows that -Dparallel-tests does not add any IT
related profiles and results in skipping all the integration tests during
verify phrase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16683) Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException
[ https://issues.apache.org/jira/browse/HADOOP-16683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal updated HADOOP-16683: Attachment: HADOOP-16683.branch-3.1.001.patch > Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped > AccessControlException > -- > > Key: HADOOP-16683 > URL: https://issues.apache.org/jira/browse/HADOOP-16683 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16683.001.patch, HADOOP-16683.002.patch, > HADOOP-16683.003.patch, HADOOP-16683.branch-3.1.001.patch, > HADOOP-16683.branch-3.2.001.patch > > > Follow up patch on HADOOP-16580. > We successfully disabled the retry in case of an AccessControlException which > has resolved some of the cases, but in other cases AccessControlException is > wrapped inside another IOException and you can only get the original > exception by calling getCause(). > Let's add this extra case as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16683) Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException
[ https://issues.apache.org/jira/browse/HADOOP-16683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal updated HADOOP-16683: Attachment: HADOOP-16683.branch-3.2.001.patch > Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped > AccessControlException > -- > > Key: HADOOP-16683 > URL: https://issues.apache.org/jira/browse/HADOOP-16683 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16683.001.patch, HADOOP-16683.002.patch, > HADOOP-16683.003.patch, HADOOP-16683.branch-3.2.001.patch > > > Follow up patch on HADOOP-16580. > We successfully disabled the retry in case of an AccessControlException which > has resolved some of the cases, but in other cases AccessControlException is > wrapped inside another IOException and you can only get the original > exception by calling getCause(). > Let's add this extra case as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-16683) Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException
[ https://issues.apache.org/jira/browse/HADOOP-16683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal reopened HADOOP-16683: - Let's backport this issue to lower branches. > Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped > AccessControlException > -- > > Key: HADOOP-16683 > URL: https://issues.apache.org/jira/browse/HADOOP-16683 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16683.001.patch, HADOOP-16683.002.patch, > HADOOP-16683.003.patch > > > Follow up patch on HADOOP-16580. > We successfully disabled the retry in case of an AccessControlException which > has resolved some of the cases, but in other cases AccessControlException is > wrapped inside another IOException and you can only get the original > exception by calling getCause(). > Let's add this extra case as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16683) Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException
[ https://issues.apache.org/jira/browse/HADOOP-16683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974304#comment-16974304 ] Adam Antal commented on HADOOP-16683: - Hi [~xkrogen]! Yes, it would make sense. Let me reopen this jira and upload two patch for branch-3.2 and branch-3.1. > Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped > AccessControlException > -- > > Key: HADOOP-16683 > URL: https://issues.apache.org/jira/browse/HADOOP-16683 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16683.001.patch, HADOOP-16683.002.patch, > HADOOP-16683.003.patch > > > Follow up patch on HADOOP-16580. > We successfully disabled the retry in case of an AccessControlException which > has resolved some of the cases, but in other cases AccessControlException is > wrapped inside another IOException and you can only get the original > exception by calling getCause(). > Let's add this extra case as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16683) Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException
[ https://issues.apache.org/jira/browse/HADOOP-16683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal updated HADOOP-16683: Target Version/s: 3.3.0, 3.1.4, 3.2.2 (was: 3.3.0) Status: Patch Available (was: Reopened) > Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped > AccessControlException > -- > > Key: HADOOP-16683 > URL: https://issues.apache.org/jira/browse/HADOOP-16683 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16683.001.patch, HADOOP-16683.002.patch, > HADOOP-16683.003.patch > > > Follow up patch on HADOOP-16580. > We successfully disabled the retry in case of an AccessControlException which > has resolved some of the cases, but in other cases AccessControlException is > wrapped inside another IOException and you can only get the original > exception by calling getCause(). > Let's add this extra case as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-16706) ITestClientUrlScheme fails for accounts which don't support HTTP
[
https://issues.apache.org/jira/browse/HADOOP-16706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota reassigned HADOOP-16706:
---
Assignee: Gabor Bota
> ITestClientUrlScheme fails for accounts which don't support HTTP
>
>
> Key: HADOOP-16706
> URL: https://issues.apache.org/jira/browse/HADOOP-16706
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Minor
>
> I'm setting up a new Storage account to Play with encryption options. I'm
> getting a test failure in
> {{testClientUrlScheme[0](org.apache.hadoop.fs.azurebfs.ITestClientUrlScheme)}}
> as it doesn't support HTTP.
> Proposed: catch, recognise 'AccountRequiresHttps' and downgrade those
> particular parameterised tests to skipped tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16709) Consider having the ability to turn off TTL in S3Guard + Authoritative mode
[ https://issues.apache.org/jira/browse/HADOOP-16709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974210#comment-16974210 ] Gabor Bota commented on HADOOP-16709: - Sure. I can do this. The general idea behind expiry was that you can set a large enough number to virtually _disable_ TTL, but sure we can add a -1 value to skip checking for expiry. > Consider having the ability to turn off TTL in S3Guard + Authoritative mode > --- > > Key: HADOOP-16709 > URL: https://issues.apache.org/jira/browse/HADOOP-16709 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Rajesh Balamohan >Priority: Minor > > Authoritative mode has TTL which is set to 15 minutes by default. However, > there are cases when we know for sure that the data wouldn't be > changed/updated. > In certain cases, AppMaster ends up spending good amount of time in getSplits > due to TTL expiry. It would be great to have an option to disable TTL (or > specify as -1 when TTL shouldn't be checked). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #763: [WIP] HADOOP-15984. Update jersey from 1.19 to 2.x
hadoop-yetus commented on issue #763: [WIP] HADOOP-15984. Update jersey from 1.19 to 2.x URL: https://github.com/apache/hadoop/pull/763#issuecomment-553863069 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 79 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 5 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 62 | Maven dependency ordering for branch | | +1 | mvninstall | 1285 | trunk passed | | +1 | compile | 1097 | trunk passed | | +1 | checkstyle | 179 | trunk passed | | +1 | mvnsite | 355 | trunk passed | | +1 | shadedclient | 1435 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 365 | trunk passed | | 0 | spotbugs | 114 | Used deprecated FindBugs config; considering switching to SpotBugs. | | 0 | findbugs | 25 | branch/hadoop-project no findbugs output file (findbugsXml.xml) | ||| _ Patch Compile Tests _ | | 0 | mvndep | 25 | Maven dependency ordering for patch | | -1 | mvninstall | 26 | hadoop-yarn-common in the patch failed. | | -1 | compile | 318 | root in the patch failed. | | -1 | javac | 318 | root in the patch failed. | | -0 | checkstyle | 170 | root: The patch generated 8 new + 631 unchanged - 24 fixed = 639 total (was 655) | | -1 | mvnsite | 29 | hadoop-yarn-common in the patch failed. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 13 | The patch has no ill-formed XML file. | | -1 | shadedclient | 207 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 43 | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common generated 11 new + 4190 unchanged - 0 fixed = 4201 total (was 4190) | | 0 | findbugs | 14 | hadoop-project has no data from findbugs | | -1 | findbugs | 42 | hadoop-common-project/hadoop-kms generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -1 | findbugs | 28 | hadoop-yarn-common in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 14 | hadoop-project in the patch passed. | | +1 | unit | 562 | hadoop-common in the patch passed. | | +1 | unit | 203 | hadoop-kms in the patch passed. | | -1 | unit | 6800 | hadoop-hdfs in the patch failed. | | -1 | unit | 268 | hadoop-hdfs-httpfs in the patch failed. | | +1 | unit | 648 | hadoop-hdfs-rbf in the patch passed. | | -1 | unit | 34 | hadoop-yarn-common in the patch failed. | | +1 | asflicense | 34 | The patch does not generate ASF License warnings. | | | | 15773 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-common-project/hadoop-kms | | | Dead store to idx in org.apache.hadoop.crypto.key.kms.server.KMS.createKey(Map) At KMS.java:org.apache.hadoop.crypto.key.kms.server.KMS.createKey(Map) At KMS.java:[line 180] | | Failed junit tests | hadoop.hdfs.web.TestWebHdfsTokens | | | hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer | | | hadoop.hdfs.web.TestWebHdfsUrl | | | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes | | | hadoop.hdfs.web.TestWebHdfsFileSystemContract | | | hadoop.fs.http.server.TestHttpFSServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-763/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/763 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle | | uname | Linux 29c75daeb139 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 73a386a | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-763/12/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-763/12/artifact/out/patch-compile-root.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-763/12/artifact/out/patch-compile-root.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-763/12/artifact/out/diff-checkstyle-root.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-763/12/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | javadoc |
[jira] [Updated] (HADOOP-16707) NPE in UGI.getCurrentUser in ITestAbfsIdentityTransformer setup
[
https://issues.apache.org/jira/browse/HADOOP-16707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-16707:
Description:
Change logs of the files point some of the blame at me here; UGI.getCurrentUser
is failing in ITestAbfsIdentityTransformer init just after a UGI.reset()
The public entry points in UGI should all call ensureInitialized().
I have no idea how this worked at all. I've been fiddling with ABFS accounts
and tests which did work now cause NPEs in UGI.
This is all test setup; I could probably fix it in the ABFS test case simply by
calling a UGI method we know calls {{ensureInitialized()}}. But fixing UGI is
the right thing to do.
was:
Change logs of the files point some of the blame at me here; UGI.getCurrentUser
is failing in ITestAbfsIdentityTransformer init just after a UGI.reset()
The public entry points in UGI should all call ensureInitialized().
I have no idea how this worked at all. I've been fiddling with ABFS accounts
and tests which did work now cause NPEs in UGI.
This is all test setup; I could probably fix it in the ABF S test case simply
by calling A UGI method we know calls {{ensureInitialized()}}. But fixing UGI
is the right thing to do.
> NPE in UGI.getCurrentUser in ITestAbfsIdentityTransformer setup
> ---
>
> Key: HADOOP-16707
> URL: https://issues.apache.org/jira/browse/HADOOP-16707
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: auth, fs/azure, security
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
>
> Change logs of the files point some of the blame at me here;
> UGI.getCurrentUser is failing in ITestAbfsIdentityTransformer init just after
> a UGI.reset()
> The public entry points in UGI should all call ensureInitialized().
> I have no idea how this worked at all. I've been fiddling with ABFS accounts
> and tests which did work now cause NPEs in UGI.
> This is all test setup; I could probably fix it in the ABFS test case simply
> by calling a UGI method we know calls {{ensureInitialized()}}. But fixing UGI
> is the right thing to do.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16709) Consider having the ability to turn off TTL in S3Guard + Authoritative mode
[ https://issues.apache.org/jira/browse/HADOOP-16709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16709: Parent: HADOOP-15620 Issue Type: Sub-task (was: Improvement) > Consider having the ability to turn off TTL in S3Guard + Authoritative mode > --- > > Key: HADOOP-16709 > URL: https://issues.apache.org/jira/browse/HADOOP-16709 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Rajesh Balamohan >Priority: Minor > > Authoritative mode has TTL which is set to 15 minutes by default. However, > there are cases when we know for sure that the data wouldn't be > changed/updated. > In certain cases, AppMaster ends up spending good amount of time in getSplits > due to TTL expiry. It would be great to have an option to disable TTL (or > specify as -1 when TTL shouldn't be checked). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16709) Consider having the ability to turn off TTL in S3Guard + Authoritative mode
[ https://issues.apache.org/jira/browse/HADOOP-16709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974112#comment-16974112 ] Steve Loughran commented on HADOOP-16709: - Moved under HADOOP-15620. [~rajesh.balamohan] -can you add things under there so they don't get lost, tag version affected, etc. Thanks. +[~gabor.bota] > Consider having the ability to turn off TTL in S3Guard + Authoritative mode > --- > > Key: HADOOP-16709 > URL: https://issues.apache.org/jira/browse/HADOOP-16709 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Rajesh Balamohan >Priority: Minor > > Authoritative mode has TTL which is set to 15 minutes by default. However, > there are cases when we know for sure that the data wouldn't be > changed/updated. > In certain cases, AppMaster ends up spending good amount of time in getSplits > due to TTL expiry. It would be great to have an option to disable TTL (or > specify as -1 when TTL shouldn't be checked). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16709) Consider having the ability to turn off TTL in S3Guard + Authoritative mode
[ https://issues.apache.org/jira/browse/HADOOP-16709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16709: Affects Version/s: 3.3.0 > Consider having the ability to turn off TTL in S3Guard + Authoritative mode > --- > > Key: HADOOP-16709 > URL: https://issues.apache.org/jira/browse/HADOOP-16709 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Rajesh Balamohan >Priority: Minor > > Authoritative mode has TTL which is set to 15 minutes by default. However, > there are cases when we know for sure that the data wouldn't be > changed/updated. > In certain cases, AppMaster ends up spending good amount of time in getSplits > due to TTL expiry. It would be great to have an option to disable TTL (or > specify as -1 when TTL shouldn't be checked). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
