[
https://issues.apache.org/jira/browse/HADOOP-17694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Ravi M V updated HADOOP-17694:
-----------------------------------
Description:
If we set the default auth type as Custom and then decided to use OAuth type
for some select storage accounts then the fs initialization for storage
accounts with Oauth type authn fails.
Steps to recreate
{code:java}
conf.set("fs.abfss.impl",
"org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem")
conf.set("fs.azure.account.auth.type", "Custom")
conf.set("fs.azure.account.oauth.provider.type", "xxx.yyy.zzz.ADTokenAdaptee")
conf.set("fs.azure.account.auth.type.abctest.dfs.core.windows.net", "OAuth")
conf.set("fs.azure.account.oauth.provider.type.abctest.dfs.core.windows.net",
"org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider")
val fs = FileSystem.get(
new
URI("abfss://conatiner...@abctest.dfs.core.windows.net/arion-scribe-de-dev"),
conf)
{code}
Error: java.lang.RuntimeException: class xxx.yyy.zzz.ADTokenAdaptee not
org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider
Cause:
In [AbfsConfiguration.
getTokenProvider|https://github.com/apache/hadoop/blob/aa96f1871bfd858f9bac59cf2a81ec470da649af/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java#L540]
, after evaluating the auth type as OAuth, the program proceeds to get the
implementing class using property `fs.azure.account.oauth.provider.type`,
while doing so the first
[step|https://github.com/apache/hadoop/blob/aa96f1871bfd858f9bac59cf2a81ec470da649af/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java#L321]
is to get the default auth class, which in our case is Custom. Here the
problem is Default Auth class is CustomTokenProviderAdaptee implementation and
not implementing AccessTokenProvider.class, hence program would fail.
proposed solution:
In the getClass function in AbfsConfiguration, we split the logic and not use
the default value property
{code:java}
public <U> Class<? extends U> getClass(String name, Class<? extends U>
defaultValue, Class<U> xface) {
Class<? extends U> klass = rawConfig.getClass(accountConf(name),
null, xface);
if(klass!=null){
return klass;
}else{
return rawConfig.getClass(name, defaultValue, xface);
}
}
{code}
was:
If we set the default auth type as Custom and then decided to use OAuth type
for some select storage accounts then the fs initialization for storage
accounts with Oauth type authn fails.
Steps to recreate
{code:java}
conf.set("fs.abfss.impl",
"org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem")
conf.set("fs.azure.account.auth.type", "Custom")
conf.set("fs.azure.account.oauth.provider.type", "xxx.yyy.zzz.ADTokenAdaptee")
conf.set("fs.azure.account.auth.type.abctest.dfs.core.windows.net", "OAuth")
conf.set("fs.azure.account.oauth.provider.type.abctest.dfs.core.windows.net",
"org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider")
val fs = FileSystem.get(
new
URI("abfs://conatiner...@abctest.dfs.core.windows.net/arion-scribe-de-dev"),
conf)
{code}
Error: java.lang.RuntimeException: class xxx.yyy.zzz.ADTokenAdaptee not
org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider
Cause:
In [AbfsConfiguration.
getTokenProvider|https://github.com/apache/hadoop/blob/aa96f1871bfd858f9bac59cf2a81ec470da649af/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java#L540]
, after evaluating the auth type as OAuth, the program proceeds to get the
implementing class using property `fs.azure.account.oauth.provider.type`,
while doing so the first
[step|https://github.com/apache/hadoop/blob/aa96f1871bfd858f9bac59cf2a81ec470da649af/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java#L321]
is to get the default auth class, which in our case is Custom. Here the
problem is Default Auth class is CustomTokenProviderAdaptee implementation and
not implementing AccessTokenProvider.class, hence program would fail.
proposed solution:
In the getClass function in AbfsConfiguration, we split the logic and not use
the default value property
{code:java}
public <U> Class<? extends U> getClass(String name, Class<? extends U>
defaultValue, Class<U> xface) {
Class<? extends U> klass = rawConfig.getClass(accountConf(name),
null, xface);
if(klass!=null){
return klass;
}else{
return rawConfig.getClass(name, defaultValue, xface);
}
}
{code}
> abfs: Unable to use OAuth authentication at storage account level if the
> default authn type is Custom
> -----------------------------------------------------------------------------------------------------
>
> Key: HADOOP-17694
> URL: https://issues.apache.org/jira/browse/HADOOP-17694
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, tools
> Affects Versions: 3.3.0
> Reporter: Arun Ravi M V
> Priority: Major
>
> If we set the default auth type as Custom and then decided to use OAuth type
> for some select storage accounts then the fs initialization for storage
> accounts with Oauth type authn fails.
> Steps to recreate
> {code:java}
> conf.set("fs.abfss.impl",
> "org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem")
> conf.set("fs.azure.account.auth.type", "Custom")
> conf.set("fs.azure.account.oauth.provider.type", "xxx.yyy.zzz.ADTokenAdaptee")
> conf.set("fs.azure.account.auth.type.abctest.dfs.core.windows.net", "OAuth")
> conf.set("fs.azure.account.oauth.provider.type.abctest.dfs.core.windows.net",
> "org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider")
> val fs = FileSystem.get(
> new
> URI("abfss://conatiner...@abctest.dfs.core.windows.net/arion-scribe-de-dev"),
> conf)
> {code}
> Error: java.lang.RuntimeException: class xxx.yyy.zzz.ADTokenAdaptee not
> org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider
> Cause:
> In [AbfsConfiguration.
> getTokenProvider|https://github.com/apache/hadoop/blob/aa96f1871bfd858f9bac59cf2a81ec470da649af/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java#L540]
> , after evaluating the auth type as OAuth, the program proceeds to get the
> implementing class using property `fs.azure.account.oauth.provider.type`,
> while doing so the first
> [step|https://github.com/apache/hadoop/blob/aa96f1871bfd858f9bac59cf2a81ec470da649af/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java#L321]
> is to get the default auth class, which in our case is Custom. Here the
> problem is Default Auth class is CustomTokenProviderAdaptee implementation
> and not implementing AccessTokenProvider.class, hence program would fail.
> proposed solution:
> In the getClass function in AbfsConfiguration, we split the logic and not
> use the default value property
> {code:java}
> public <U> Class<? extends U> getClass(String name, Class<? extends U>
> defaultValue, Class<U> xface) {
> Class<? extends U> klass = rawConfig.getClass(accountConf(name),
> null, xface);
>
> if(klass!=null){
> return klass;
> }else{
> return rawConfig.getClass(name, defaultValue, xface);
> }
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
