[jira] [Commented] (HADOOP-15757) ABFS: remove dependency on common-codec Base64
[ https://issues.apache.org/jira/browse/HADOOP-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16676352#comment-16676352 ] Sunil Govindan commented on HADOOP-15757: - Updated fixed version to 3.2.0 as HADOOP-15407 branch is merged to trunk > ABFS: remove dependency on common-codec Base64 > -- > > Key: HADOOP-15757 > URL: https://issues.apache.org/jira/browse/HADOOP-15757 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Da Zhou >Assignee: Da Zhou >Priority: Major > Fix For: 3.2.0 > > Attachments: HADOOP-15757-HADOOP-15407-001.patch, > HADOOP-15757-HADOOP-15407-002.patch > > > Currently ABFS relies on common-codec Base64, because different versions of > common-codec are widely used and some are missing the methods needed by ABFS, > it cause lots of "no such method" exception in customer's env, hence we > decide to add util for Base64 to avoid such issues in future. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15757) ABFS: remove dependency on common-codec Base64
[ https://issues.apache.org/jira/browse/HADOOP-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624945#comment-16624945 ] Hudson commented on HADOOP-15757: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15037 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15037/]) HADOOP-15757. ABFS: remove dependency on common-codec Base64. (tmarq: rev b4c23043d38325aad8af4488caca115eae789f07) * (edit) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java * (edit) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/Base64StringConfigurationBasicValidator.java * (edit) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/diagnostics/TestConfigurationValidators.java * (edit) hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml * (edit) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAbfsConfigurationFieldsValidation.java * (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/Base64.java * (edit) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/SharedKeyCredentials.java > ABFS: remove dependency on common-codec Base64 > -- > > Key: HADOOP-15757 > URL: https://issues.apache.org/jira/browse/HADOOP-15757 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Da Zhou >Assignee: Da Zhou >Priority: Major > Attachments: HADOOP-15757-HADOOP-15407-001.patch, > HADOOP-15757-HADOOP-15407-002.patch > > > Currently ABFS relies on common-codec Base64, because different versions of > common-codec are widely used and some are missing the methods needed by ABFS, > it cause lots of "no such method" exception in customer's env, hence we > decide to add util for Base64 to avoid such issues in future. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15757) ABFS: remove dependency on common-codec Base64
[
https://issues.apache.org/jira/browse/HADOOP-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615487#comment-16615487
]
Hadoop QA commented on HADOOP-15757:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} HADOOP-15407 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
24s{color} | {color:green} HADOOP-15407 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} HADOOP-15407 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} HADOOP-15407 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} HADOOP-15407 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 58s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} HADOOP-15407 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} HADOOP-15407 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 31s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
7s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HADOOP-15757 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12939792/HADOOP-15757-HADOOP-15407-002.patch
|
| Optional Tests | dupname asflicense xml compile javac javadoc
mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux ffc45777f512 4.4.0-134-generic #160~14.04.1-Ubuntu SMP Fri Aug
17 11:07:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | HADOOP-15407 / 39bacd6 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15197/testReport/ |
| Max. process+thread count | 345 (vs. ulimit of 1) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15197/console |
| Powered by | Apache Yetus 0.8.0 http
[jira] [Commented] (HADOOP-15757) ABFS: remove dependency on common-codec Base64
[
https://issues.apache.org/jira/browse/HADOOP-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615464#comment-16615464
]
Thomas Marquardt commented on HADOOP-15757:
---
+1 on the 002 patch. As long as Yetus gives a +1, we can push it.
All tests pass for me too:
{code:java}
mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify
Tests run: 35, Failures: 0, Errors: 0, Skipped: 0
Tests run: 269, Failures: 0, Errors: 0, Skipped: 182
Tests run: 167, Failures: 0, Errors: 0, Skipped: 27
Total time: 04:26 min (Wall Clock)
{code}
> ABFS: remove dependency on common-codec Base64
> --
>
> Key: HADOOP-15757
> URL: https://issues.apache.org/jira/browse/HADOOP-15757
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Da Zhou
>Assignee: Da Zhou
>Priority: Major
> Attachments: HADOOP-15757-HADOOP-15407-001.patch,
> HADOOP-15757-HADOOP-15407-002.patch
>
>
> Currently ABFS relies on common-codec Base64, because different versions of
> common-codec are widely used and some are missing the methods needed by ABFS,
> it cause lots of "no such method" exception in customer's env, hence we
> decide to add util for Base64 to avoid such issues in future.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15757) ABFS: remove dependency on common-codec Base64
[
https://issues.apache.org/jira/browse/HADOOP-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615438#comment-16615438
]
Hadoop QA commented on HADOOP-15757:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 19m
57s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} HADOOP-15407 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 29m
58s{color} | {color:green} HADOOP-15407 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} HADOOP-15407 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} HADOOP-15407 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} HADOOP-15407 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 45s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} HADOOP-15407 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} HADOOP-15407 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 15s{color} | {color:orange} hadoop-tools/hadoop-azure: The patch generated 1
new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 43s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
14s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 79m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HADOOP-15757 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12939782/HADOOP-15757-HADOOP-15407-001.patch
|
| Optional Tests | dupname asflicense xml compile javac javadoc
mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 00fbfdcd499d 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10
07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | HADOOP-15407 / 39bacd6 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15196/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15196/testReport/ |
| Max. process+thread count | 409 (vs. ulimit o
[jira] [Commented] (HADOOP-15757) ABFS: remove dependency on common-codec Base64
[ https://issues.apache.org/jira/browse/HADOOP-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615435#comment-16615435 ] Da Zhou commented on HADOOP-15757: -- [~tmarquardt]Thanks for the review. Submitting HADOOP-15757-HADOOP-15407-002.patch: - Base64.java L33: resolved - TestConfigurationValidators.java, L24: resolved. > ABFS: remove dependency on common-codec Base64 > -- > > Key: HADOOP-15757 > URL: https://issues.apache.org/jira/browse/HADOOP-15757 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Da Zhou >Assignee: Da Zhou >Priority: Major > Attachments: HADOOP-15757-HADOOP-15407-001.patch, > HADOOP-15757-HADOOP-15407-002.patch > > > Currently ABFS relies on common-codec Base64, because different versions of > common-codec are widely used and some are missing the methods needed by ABFS, > it cause lots of "no such method" exception in customer's env, hence we > decide to add util for Base64 to avoid such issues in future. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15757) ABFS: remove dependency on common-codec Base64
[
https://issues.apache.org/jira/browse/HADOOP-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615380#comment-16615380
]
Thomas Marquardt commented on HADOOP-15757:
---
LGTM, couple minor comments:
*Base64.java*:
L33: {{byte DECODE_64[]}} should be {{byte[] DECODE_64}}.
*TestConfigurationValidators.java*:
L24: imports should be in alphabetical order, so
org.apache.hadoop.fs.azurebfs.utils.Base64 is below
org.apache.hadoop.fs.azurebfs.contracts.exceptions.InvalidConfigurationValueException
Otherwise, +1 from me. After we move to trunk, let's move this to
hadoop-common. We've actually used this class for years, so we no it is
correct and very stable.
> ABFS: remove dependency on common-codec Base64
> --
>
> Key: HADOOP-15757
> URL: https://issues.apache.org/jira/browse/HADOOP-15757
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Da Zhou
>Assignee: Da Zhou
>Priority: Major
> Attachments: HADOOP-15757-HADOOP-15407-001.patch
>
>
> Currently ABFS relies on common-codec Base64, because different versions of
> common-codec are widely used and some are missing the methods needed by ABFS,
> it cause lots of "no such method" exception in customer's env, hence we
> decide to add util for Base64 to avoid such issues in future.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15757) ABFS: remove dependency on common-codec Base64
[ https://issues.apache.org/jira/browse/HADOOP-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615337#comment-16615337 ] Da Zhou commented on HADOOP-15757: -- ABFS tests passed my US West account Tests results: Tests run: 35, Failures: 0, Errors: 0, Skipped: 0 Tests run: 269, Failures: 0, Errors: 0, Skipped: 31 Tests run: 167, Failures: 0, Errors: 0, Skipped: 27 > ABFS: remove dependency on common-codec Base64 > -- > > Key: HADOOP-15757 > URL: https://issues.apache.org/jira/browse/HADOOP-15757 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Da Zhou >Assignee: Da Zhou >Priority: Major > Attachments: HADOOP-15757-HADOOP-15407-001.patch > > > Currently ABFS relies on common-codec Base64, because different versions of > common-codec are widely used and some are missing the methods needed by ABFS, > it cause lots of "no such method" exception, hence we decide to add util for > Base64 to avoid such issues in future. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
