[jira] [Commented] (HADOOP-14565) Azure: Add Authorization support to ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17803065#comment-17803065 ] Shilun Fan commented on HADOOP-14565: - Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a blocker. Retarget 3.5.0. > Azure: Add Authorization support to ADLS > > > Key: HADOOP-14565 > URL: https://issues.apache.org/jira/browse/HADOOP-14565 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/adl >Affects Versions: 2.8.0 >Reporter: Ryan Waters >Assignee: Sivaguru Sankaridurg >Priority: Major > Attachments: > HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch > > > This task is meant to add an Authorizer interface to be used by the ADLS > driver in a similar way to the one used by WASB. The primary difference in > functionality being that the implementation of this Authorizer will be > provided by an external jar. This class will be specified through > configuration using "adl.external.authorization.class". > If this configuration is provided, an instance of the provided class will be > created and all file system calls will be passed through the authorizer, > allowing implementations to determine if the file path and access type > (create, open, delete, etc.) being requested is valid. If the requested > implementation class is not found or it fails to initialize, it will fail > initialization of the ADL driver. If no configuration is provided, calls to > the authorizer will be skipped and the driver will behave as it did > previously. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14565) Azure: Add Authorization support to ADLS
[
https://issues.apache.org/jira/browse/HADOOP-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608738#comment-16608738
]
Hadoop QA commented on HADOOP-14565:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HADOOP-14565 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-14565 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880108/HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15164/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Azure: Add Authorization support to ADLS
>
>
> Key: HADOOP-14565
> URL: https://issues.apache.org/jira/browse/HADOOP-14565
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/adl
>Affects Versions: 2.8.0
>Reporter: Ryan Waters
>Assignee: Sivaguru Sankaridurg
>Priority: Major
> Attachments:
> HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch
>
>
> This task is meant to add an Authorizer interface to be used by the ADLS
> driver in a similar way to the one used by WASB. The primary difference in
> functionality being that the implementation of this Authorizer will be
> provided by an external jar. This class will be specified through
> configuration using "adl.external.authorization.class".
> If this configuration is provided, an instance of the provided class will be
> created and all file system calls will be passed through the authorizer,
> allowing implementations to determine if the file path and access type
> (create, open, delete, etc.) being requested is valid. If the requested
> implementation class is not found or it fails to initialize, it will fail
> initialization of the ADL driver. If no configuration is provided, calls to
> the authorizer will be skipped and the driver will behave as it did
> previously.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14565) Azure: Add Authorization support to ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608721#comment-16608721 ] Sunil Govindan commented on HADOOP-14565: - Hi [~rywater] [~ASikaria] Are we targeting this for 3.2.0. There are no much progress on this recently. Deferring to 3.3 if there are no immediate plans. > Azure: Add Authorization support to ADLS > > > Key: HADOOP-14565 > URL: https://issues.apache.org/jira/browse/HADOOP-14565 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/adl >Affects Versions: 2.8.0 >Reporter: Ryan Waters >Assignee: Sivaguru Sankaridurg >Priority: Major > Attachments: > HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch > > > This task is meant to add an Authorizer interface to be used by the ADLS > driver in a similar way to the one used by WASB. The primary difference in > functionality being that the implementation of this Authorizer will be > provided by an external jar. This class will be specified through > configuration using "adl.external.authorization.class". > If this configuration is provided, an instance of the provided class will be > created and all file system calls will be passed through the authorizer, > allowing implementations to determine if the file path and access type > (create, open, delete, etc.) being requested is valid. If the requested > implementation class is not found or it fails to initialize, it will fail > initialization of the ADL driver. If no configuration is provided, calls to > the authorizer will be skipped and the driver will behave as it did > previously. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14565) Azure: Add Authorization support to ADLS
[
https://issues.apache.org/jira/browse/HADOOP-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16276915#comment-16276915
]
genericqa commented on HADOOP-14565:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HADOOP-14565 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-14565 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880108/HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13778/console |
| Powered by | Apache Yetus 0.7.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Azure: Add Authorization support to ADLS
>
>
> Key: HADOOP-14565
> URL: https://issues.apache.org/jira/browse/HADOOP-14565
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/adl
>Affects Versions: 2.8.0
>Reporter: Ryan Waters
>Assignee: Sivaguru Sankaridurg
> Attachments:
> HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch
>
>
> This task is meant to add an Authorizer interface to be used by the ADLS
> driver in a similar way to the one used by WASB. The primary difference in
> functionality being that the implementation of this Authorizer will be
> provided by an external jar. This class will be specified through
> configuration using "adl.external.authorization.class".
> If this configuration is provided, an instance of the provided class will be
> created and all file system calls will be passed through the authorizer,
> allowing implementations to determine if the file path and access type
> (create, open, delete, etc.) being requested is valid. If the requested
> implementation class is not found or it fails to initialize, it will fail
> initialization of the ADL driver. If no configuration is provided, calls to
> the authorizer will be skipped and the driver will behave as it did
> previously.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14565) Azure: Add Authorization support to ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16276911#comment-16276911 ] Steve Loughran commented on HADOOP-14565: - Based on the experience of WASB # I'd like as realistic testing of this as possible, ideally even bringing up some minimal REST endpoint to do it. Why? Ensures that the production codepath gets tested before integration test time # add lots of logging, diagnostics and timeouts here. See #1 > Azure: Add Authorization support to ADLS > > > Key: HADOOP-14565 > URL: https://issues.apache.org/jira/browse/HADOOP-14565 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/adl >Affects Versions: 2.8.0 >Reporter: Ryan Waters >Assignee: Sivaguru Sankaridurg > Attachments: > HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch > > > This task is meant to add an Authorizer interface to be used by the ADLS > driver in a similar way to the one used by WASB. The primary difference in > functionality being that the implementation of this Authorizer will be > provided by an external jar. This class will be specified through > configuration using "adl.external.authorization.class". > If this configuration is provided, an instance of the provided class will be > created and all file system calls will be passed through the authorizer, > allowing implementations to determine if the file path and access type > (create, open, delete, etc.) being requested is valid. If the requested > implementation class is not found or it fails to initialize, it will fail > initialization of the ADL driver. If no configuration is provided, calls to > the authorizer will be skipped and the driver will behave as it did > previously. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14565) Azure: Add Authorization support to ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16186564#comment-16186564 ] Atul Sikaria commented on HADOOP-14565: --- Yes, we should move this out of 2.9. > Azure: Add Authorization support to ADLS > > > Key: HADOOP-14565 > URL: https://issues.apache.org/jira/browse/HADOOP-14565 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl >Affects Versions: 2.8.0 >Reporter: Ryan Waters >Assignee: Sivaguru Sankaridurg > Attachments: > HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch > > > This task is meant to add an Authorizer interface to be used by the ADLS > driver in a similar way to the one used by WASB. The primary difference in > functionality being that the implementation of this Authorizer will be > provided by an external jar. This class will be specified through > configuration using "adl.external.authorization.class". > If this configuration is provided, an instance of the provided class will be > created and all file system calls will be passed through the authorizer, > allowing implementations to determine if the file path and access type > (create, open, delete, etc.) being requested is valid. If the requested > implementation class is not found or it fails to initialize, it will fail > initialization of the ADL driver. If no configuration is provided, calls to > the authorizer will be skipped and the driver will behave as it did > previously. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14565) Azure: Add Authorization support to ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113215#comment-16113215 ] Atul Sikaria commented on HADOOP-14565: --- [~rywater], this is unrelated to your patch. I saw this happen on mine as well, after I did a git pull. [~vishwajeet.dusane] is looking into this. tldr version (Vishwajeet can provide more details): New ACL checks were introduced by commit 4966a6e2, and there was a recent change that is causing these tests to fail. [~vishwajeet.dusane] has opened HADOOP-14730 for this. > Azure: Add Authorization support to ADLS > > > Key: HADOOP-14565 > URL: https://issues.apache.org/jira/browse/HADOOP-14565 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl >Affects Versions: 2.8.0 >Reporter: Ryan Waters >Assignee: Sivaguru Sankaridurg > Attachments: > HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch > > > This task is meant to add an Authorizer interface to be used by the ADLS > driver in a similar way to the one used by WASB. The primary difference in > functionality being that the implementation of this Authorizer will be > provided by an external jar. This class will be specified through > configuration using "adl.external.authorization.class". > If this configuration is provided, an instance of the provided class will be > created and all file system calls will be passed through the authorizer, > allowing implementations to determine if the file path and access type > (create, open, delete, etc.) being requested is valid. If the requested > implementation class is not found or it fails to initialize, it will fail > initialization of the ADL driver. If no configuration is provided, calls to > the authorizer will be skipped and the driver will behave as it did > previously. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14565) Azure: Add Authorization support to ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113204#comment-16113204 ] Ryan Waters commented on HADOOP-14565: -- These tests fail on trunk without my patch as well. > Azure: Add Authorization support to ADLS > > > Key: HADOOP-14565 > URL: https://issues.apache.org/jira/browse/HADOOP-14565 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl >Affects Versions: 2.8.0 >Reporter: Ryan Waters >Assignee: Sivaguru Sankaridurg > Attachments: > HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch > > > This task is meant to add an Authorizer interface to be used by the ADLS > driver in a similar way to the one used by WASB. The primary difference in > functionality being that the implementation of this Authorizer will be > provided by an external jar. This class will be specified through > configuration using "adl.external.authorization.class". > If this configuration is provided, an instance of the provided class will be > created and all file system calls will be passed through the authorizer, > allowing implementations to determine if the file path and access type > (create, open, delete, etc.) being requested is valid. If the requested > implementation class is not found or it fails to initialize, it will fail > initialization of the ADL driver. If no configuration is provided, calls to > the authorizer will be skipped and the driver will behave as it did > previously. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14565) Azure: Add Authorization support to ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112930#comment-16112930 ] Ryan Waters commented on HADOOP-14565: -- Hmm. Didn't get this while running test-patch locally. Will investigate and resubmit another patch. > Azure: Add Authorization support to ADLS > > > Key: HADOOP-14565 > URL: https://issues.apache.org/jira/browse/HADOOP-14565 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl >Affects Versions: 2.8.0 >Reporter: Ryan Waters >Assignee: Sivaguru Sankaridurg > Attachments: > HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch > > > This task is meant to add an Authorizer interface to be used by the ADLS > driver in a similar way to the one used by WASB. The primary difference in > functionality being that the implementation of this Authorizer will be > provided by an external jar. This class will be specified through > configuration using "adl.external.authorization.class". > If this configuration is provided, an instance of the provided class will be > created and all file system calls will be passed through the authorizer, > allowing implementations to determine if the file path and access type > (create, open, delete, etc.) being requested is valid. If the requested > implementation class is not found or it fails to initialize, it will fail > initialization of the ADL driver. If no configuration is provided, calls to > the authorizer will be skipped and the driver will behave as it did > previously. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14565) Azure: Add Authorization support to ADLS
[
https://issues.apache.org/jira/browse/HADOOP-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112066#comment-16112066
]
Hadoop QA commented on HADOOP-14565:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 10s{color} | {color:orange} hadoop-tools/hadoop-azure-datalake: The patch
generated 27 new + 0 unchanged - 0 fixed = 27 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 33s{color}
| {color:red} hadoop-azure-datalake in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 21s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.adl.TestGetFileStatus |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14565 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880108/HADOOP_14565__Added_authorizer_functionality_to_ADL_driver.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux d9d8041a8487 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 79df1e7 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12935/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure-datalake.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12935/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-azure-datalake.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12935/testReport/ |
| modules | C: hadoop-tools/hadoop-azure-datalake U:
hadoop-tools/hadoop-azure-datalake |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12935/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Azure: Add Authorization support to ADLS
>
>
> Key: HADOOP-14565
>
