[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868312#comment-17868312
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2247374679
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 19s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 11s | | trunk passed |
| +1 :green_heart: | compile | 0m 22s | | trunk passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | compile | 0m 22s | | trunk passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | checkstyle | 0m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 25s | | trunk passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javadoc | 0m 22s | | trunk passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | spotbugs | 0m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 8s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 18s | | the patch passed |
| +1 :green_heart: | compile | 0m 20s | | the patch passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javac | 0m 20s | | the patch passed |
| +1 :green_heart: | compile | 0m 16s | | the patch passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | javac | 0m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 12s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | spotbugs | 0m 42s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 57s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 25s | | The patch does not
generate ASF License warnings. |
| | | 81m 27s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.46 ServerAPI=1.46 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/15/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4262 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux dd6531721ab0 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17
09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b32b2218e70a8ec22762ad61e61912dd6a988823 |
| Default Java | Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/15/testReport/ |
| Max. process+thread count | 551 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/15/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Azure Token provider requires tenant and client IDs despite being
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868299#comment-17868299
]
ASF GitHub Bot commented on HADOOP-18542:
-
CLevasseur commented on code in PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#discussion_r1689341481
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAccountConfiguration.java:
##
@@ -444,6 +439,30 @@ private static void testMissingConfigKey(final
AbfsConfiguration abfsConf,
() -> abfsConf.getTokenProvider().getClass().getTypeName(;
}
+ @Test
+ public void testClientAndTenantIdOptionalWhenUsingMsiTokenProvider() throws
Throwable {
+ final String accountName = "account";
+ final Configuration conf = new Configuration();
+ final AbfsConfiguration abfsConf = new AbfsConfiguration(conf,
accountName);
+
+ final String accountNameSuffix = "." + abfsConf.getAccountName();
+ String authKey = FS_AZURE_ACCOUNT_AUTH_TYPE_PROPERTY_NAME +
accountNameSuffix;
+ String providerClassKey = "";
+ String providerClassValue = "";
+
+ providerClassKey = FS_AZURE_ACCOUNT_TOKEN_PROVIDER_TYPE_PROPERTY_NAME +
accountNameSuffix;
+ providerClassValue = TEST_OAUTH_MSI_TOKEN_PROVIDER_CLASS_CONFIG;
+
+ abfsConf.set(authKey, AuthType.OAuth.toString());
+ abfsConf.set(providerClassKey, providerClassValue);
+
+ AccessTokenProvider tokenProviderTypeName = abfsConf.getTokenProvider();
+ // Test that we managed to instantiate an MsiTokenProvider without
having to define the tenant and client ID.
+ // Those 2 fields are optional as they can automatically be determined
by the Azure Metadata service when
+ // running on an Azure VM.
+
Assertions.assertThat(tokenProviderTypeName).isInstanceOf(MsiTokenProvider.class);
Review Comment:
done
> Azure Token provider requires tenant and client IDs despite being optional
> --
>
> Key: HADOOP-18542
> URL: https://issues.apache.org/jira/browse/HADOOP-18542
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, hadoop-thirdparty
>Affects Versions: 3.3.2, 3.3.3, 3.3.4
>Reporter: Carl
>Priority: Major
> Labels: pull-request-available
>
> The `AbfsConfiguration` class requires that we provide a tenant and client ID
> when using the `MsiTokenProvider` class to fetch an authentication token. The
> bug is that those fields are not required by the Azure API, which can infer
> those fields when the call is made from an Azure instance.
> The fix is to make tenant and client ID optional when getting an Azure token
> from the Azure Metadata Service.
> A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262]
> The bug was introduced with HADOOP-17725
> ([https://github.com/apache/hadoop/pull/3041/files])
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868259#comment-17868259
]
ASF GitHub Bot commented on HADOOP-18542:
-
anujmodi2021 commented on code in PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#discussion_r1689199461
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAccountConfiguration.java:
##
@@ -444,6 +439,30 @@ private static void testMissingConfigKey(final
AbfsConfiguration abfsConf,
() -> abfsConf.getTokenProvider().getClass().getTypeName(;
}
+ @Test
+ public void testClientAndTenantIdOptionalWhenUsingMsiTokenProvider() throws
Throwable {
+ final String accountName = "account";
+ final Configuration conf = new Configuration();
+ final AbfsConfiguration abfsConf = new AbfsConfiguration(conf,
accountName);
+
+ final String accountNameSuffix = "." + abfsConf.getAccountName();
+ String authKey = FS_AZURE_ACCOUNT_AUTH_TYPE_PROPERTY_NAME +
accountNameSuffix;
+ String providerClassKey = "";
+ String providerClassValue = "";
+
+ providerClassKey = FS_AZURE_ACCOUNT_TOKEN_PROVIDER_TYPE_PROPERTY_NAME +
accountNameSuffix;
+ providerClassValue = TEST_OAUTH_MSI_TOKEN_PROVIDER_CLASS_CONFIG;
+
+ abfsConf.set(authKey, AuthType.OAuth.toString());
+ abfsConf.set(providerClassKey, providerClassValue);
+
+ AccessTokenProvider tokenProviderTypeName = abfsConf.getTokenProvider();
+ // Test that we managed to instantiate an MsiTokenProvider without
having to define the tenant and client ID.
+ // Those 2 fields are optional as they can automatically be determined
by the Azure Metadata service when
+ // running on an Azure VM.
+
Assertions.assertThat(tokenProviderTypeName).isInstanceOf(MsiTokenProvider.class);
Review Comment:
Nit: Add a description to the assertions so that if it fails, user know the
expectation. Something like:
`Assertions.assertThat(tokenProviderTypeName).decribedAs("Token Provider
Should be MsiTokenProvider").isInstanceOf(MsiTokenProvider.class);`
> Azure Token provider requires tenant and client IDs despite being optional
> --
>
> Key: HADOOP-18542
> URL: https://issues.apache.org/jira/browse/HADOOP-18542
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, hadoop-thirdparty
>Affects Versions: 3.3.2, 3.3.3, 3.3.4
>Reporter: Carl
>Priority: Major
> Labels: pull-request-available
>
> The `AbfsConfiguration` class requires that we provide a tenant and client ID
> when using the `MsiTokenProvider` class to fetch an authentication token. The
> bug is that those fields are not required by the Azure API, which can infer
> those fields when the call is made from an Azure instance.
> The fix is to make tenant and client ID optional when getting an Azure token
> from the Azure Metadata Service.
> A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262]
> The bug was introduced with HADOOP-17725
> ([https://github.com/apache/hadoop/pull/3041/files])
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868258#comment-17868258 ] ASF GitHub Bot commented on HADOOP-18542: - anujmodi2021 commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2246988401 > I've rebased the branch onto the latest trunk, you should be good to run the integration tests Thanks @CLevasseur I have ran tests and added result above. The failures reported are known and we have a PR already out that fixes those tests. This is good to merge from my side. I see there are some PR checks failing. Might be intermittent, you can make a small commit to re-trigger that. > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868257#comment-17868257
]
ASF GitHub Bot commented on HADOOP-18542:
-
anujmodi2021 commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2246985287
--
AGGREGATED TEST RESULT
HNS-OAuth
[ERROR]
testBackoffRetryMetrics(org.apache.hadoop.fs.azurebfs.services.TestAbfsRestOperation)
Time elapsed: 3.604 s <<< ERROR!
[ERROR]
testReadFooterMetrics(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.503 s <<< ERROR!
[ERROR]
testMetricWithIdlePeriod(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.371 s <<< ERROR!
[ERROR]
testReadFooterMetricsWithParquetAndNonParquet(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.372 s <<< ERROR!
[ERROR] Tests run: 143, Failures: 0, Errors: 1, Skipped: 2
[ERROR] Tests run: 626, Failures: 0, Errors: 3, Skipped: 76
[WARNING] Tests run: 414, Failures: 0, Errors: 0, Skipped: 57
HNS-SharedKey
[ERROR]
testBackoffRetryMetrics(org.apache.hadoop.fs.azurebfs.services.TestAbfsRestOperation)
Time elapsed: 3.124 s <<< ERROR!
[ERROR]
testReadFooterMetrics(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.026 s <<< ERROR!
[ERROR]
testMetricWithIdlePeriod(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 0.905 s <<< ERROR!
[ERROR]
testReadFooterMetricsWithParquetAndNonParquet(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.012 s <<< ERROR!
[ERROR] Tests run: 143, Failures: 0, Errors: 1, Skipped: 3
[ERROR] Tests run: 626, Failures: 0, Errors: 3, Skipped: 28
[WARNING] Tests run: 414, Failures: 0, Errors: 0, Skipped: 44
NonHNS-SharedKey
[ERROR]
testBackoffRetryMetrics(org.apache.hadoop.fs.azurebfs.services.TestAbfsRestOperation)
Time elapsed: 3.484 s <<< ERROR!
[ERROR]
testReadFooterMetrics(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.076 s <<< ERROR!
[ERROR]
testMetricWithIdlePeriod(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.079 s <<< ERROR!
[ERROR]
testReadFooterMetricsWithParquetAndNonParquet(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.028 s <<< ERROR!
[ERROR] Tests run: 143, Failures: 0, Errors: 1, Skipped: 9
[ERROR] Tests run: 610, Failures: 0, Errors: 3, Skipped: 268
[WARNING] Tests run: 414, Failures: 0, Errors: 0, Skipped: 47
AppendBlob-HNS-OAuth
[ERROR]
testBackoffRetryMetrics(org.apache.hadoop.fs.azurebfs.services.TestAbfsRestOperation)
Time elapsed: 3.495 s <<< ERROR!
[ERROR]
testReadFooterMetrics(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.195 s <<< ERROR!
[ERROR]
testMetricWithIdlePeriod(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.239 s <<< ERROR!
[ERROR]
testReadFooterMetricsWithParquetAndNonParquet(org.apache.hadoop.fs.azurebfs.ITestAbfsReadFooterMetrics)
Time elapsed: 1.205 s <<< ERROR!
[ERROR] Tests run: 143, Failures: 0, Errors: 1, Skipped: 2
[ERROR] Tests run: 626, Failures: 0, Errors: 3, Skipped: 78
[WARNING] Tests run: 414, Failures: 0, Errors: 0, Skipped: 81
Time taken: 55 mins 19 secs.
> Azure Token provider requires tenant and client IDs despite being optional
> --
>
> Key: HADOOP-18542
> URL: https://issues.apache.org/jira/browse/HADOOP-18542
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, hadoop-thirdparty
>Affects Versions: 3.3.2, 3.3.3, 3.3.4
>Reporter: Carl
>Priority: Major
> Labels: pull-request-available
>
> The `AbfsConfiguration` class requires that we provide a tenant and client ID
> when using the `MsiTokenProvider` class to fetch an authentication token. The
> bug is that those fields are not required by the Azure API, which can infer
> those fields when the call is made from an Azure instance.
> The fix is to make tenant and client ID optional when getting an Azure token
> from the Azure Metadata Service.
> A fix
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868031#comment-17868031
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2245016042
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 35s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | spotbugs | 1m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 37m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | spotbugs | 1m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 34m 51s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 38s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 135m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4262 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 18a119403eaf 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a602b81fc2638a75a52df220773e9de9a3c65550 |
| Default Java | Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/14/testReport/ |
| Max. process+thread count | 703 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/14/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Azure Token provider requires tenant and client IDs despite being optional
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868028#comment-17868028
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2244999777
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 30s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 49s | | trunk passed |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | checkstyle | 0m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javadoc | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | spotbugs | 1m 13s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | spotbugs | 1m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 59s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 22s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 38s | | The patch does not
generate ASF License warnings. |
| | | 127m 1s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.46 ServerAPI=1.46 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4262 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 497df83b6f68 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17
09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a602b81fc2638a75a52df220773e9de9a3c65550 |
| Default Java | Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/13/testReport/ |
| Max. process+thread count | 565 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/13/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Azure Token provider requires tenant and client IDs despite being
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868007#comment-17868007
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2244755461
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 7m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 0m 20s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| -1 :x: | compile | 0m 21s |
[/branch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/branch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in trunk failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | compile | 0m 21s |
[/branch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/branch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt)
| hadoop-azure in trunk failed with JDK Private
Build-1.8.0_412-8u412-ga-1~20.04.1-b08. |
| -0 :warning: | checkstyle | 0m 19s |
[/buildtool-branch-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/buildtool-branch-checkstyle-hadoop-tools_hadoop-azure.txt)
| The patch fails to run checkstyle in hadoop-azure |
| -1 :x: | mvnsite | 0m 22s |
[/branch-mvnsite-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/branch-mvnsite-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in trunk failed. |
| -1 :x: | javadoc | 0m 20s |
[/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in trunk failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | javadoc | 0m 21s |
[/branch-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt)
| hadoop-azure in trunk failed with JDK Private
Build-1.8.0_412-8u412-ga-1~20.04.1-b08. |
| -1 :x: | spotbugs | 0m 22s |
[/branch-spotbugs-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/branch-spotbugs-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in trunk failed. |
| +1 :green_heart: | shadedclient | 2m 24s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 21s |
[/patch-mvninstall-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/patch-mvninstall-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch failed. |
| -1 :x: | compile | 0m 19s |
[/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in the patch failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | javac | 0m 19s |
[/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/10/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in the patch failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | compile | 3m 56s |
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868006#comment-17868006
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2244747625
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 7m 54s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 0m 19s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| -1 :x: | compile | 0m 22s |
[/branch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/branch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in trunk failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | compile | 0m 22s |
[/branch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/branch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt)
| hadoop-azure in trunk failed with JDK Private
Build-1.8.0_412-8u412-ga-1~20.04.1-b08. |
| -0 :warning: | checkstyle | 0m 19s |
[/buildtool-branch-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/buildtool-branch-checkstyle-hadoop-tools_hadoop-azure.txt)
| The patch fails to run checkstyle in hadoop-azure |
| -1 :x: | mvnsite | 0m 22s |
[/branch-mvnsite-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/branch-mvnsite-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in trunk failed. |
| -1 :x: | javadoc | 0m 22s |
[/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in trunk failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | javadoc | 3m 55s |
[/branch-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt)
| hadoop-azure in trunk failed with JDK Private
Build-1.8.0_412-8u412-ga-1~20.04.1-b08. |
| -1 :x: | spotbugs | 0m 22s |
[/branch-spotbugs-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/branch-spotbugs-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in trunk failed. |
| -1 :x: | shadedclient | 6m 11s | | branch has errors when building
and testing our client artifacts. |
_ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 21s |
[/patch-mvninstall-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/patch-mvninstall-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch failed. |
| -1 :x: | compile | 0m 48s |
[/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in the patch failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | javac | 0m 48s |
[/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/9/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in the patch failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | compile | 0m 21s |
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868005#comment-17868005
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2244746137
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 0m 23s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| -1 :x: | compile | 0m 23s |
[/branch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/branch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in trunk failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | compile | 0m 23s |
[/branch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/branch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt)
| hadoop-azure in trunk failed with JDK Private
Build-1.8.0_412-8u412-ga-1~20.04.1-b08. |
| -0 :warning: | checkstyle | 0m 6s |
[/buildtool-branch-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/buildtool-branch-checkstyle-hadoop-tools_hadoop-azure.txt)
| The patch fails to run checkstyle in hadoop-azure |
| -1 :x: | mvnsite | 0m 23s |
[/branch-mvnsite-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/branch-mvnsite-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in trunk failed. |
| -1 :x: | javadoc | 0m 23s |
[/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in trunk failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | javadoc | 0m 23s |
[/branch-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt)
| hadoop-azure in trunk failed with JDK Private
Build-1.8.0_412-8u412-ga-1~20.04.1-b08. |
| -1 :x: | spotbugs | 0m 39s |
[/branch-spotbugs-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/branch-spotbugs-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in trunk failed. |
| +1 :green_heart: | shadedclient | 2m 44s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| -1 :x: | mvninstall | 4m 1s |
[/patch-mvninstall-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/patch-mvninstall-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch failed. |
| -1 :x: | compile | 0m 14s |
[/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in the patch failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | javac | 0m 14s |
[/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/11/artifact/out/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt)
| hadoop-azure in the patch failed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2. |
| -1 :x: | compile | 0m 23s |
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868004#comment-17868004 ] ASF GitHub Bot commented on HADOOP-18542: - CLevasseur commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2244717146 I've rebased the branch onto the latest trunk, you should be good to run the integration tests > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867816#comment-17867816 ] ASF GitHub Bot commented on HADOOP-18542: - steveloughran commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2243498857 @CLevasseur if @anujmodi2021 is happy to run the integration tests, please do the rebase and I'll go with his test results. thanks > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867268#comment-17867268 ] ASF GitHub Bot commented on HADOOP-18542: - anujmodi2021 commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2238805237 > Hey @anujmodi2021, thanks for the renewed interested in this PR. We have patched a fork on our side since this PR was created. > > I have no idea how to run that test suite properly to be honest. I got inconsistent results last time I tried and was really confused as to how to run it correctly. If you have more experience with it, do you mind trying on your side? I can surely assist you on how to run the tests, but rebasing the PR to the latest trunk is something you only have to do. This PR anyway can't be merged without resolving conflicts... Once you have rebased the PR, I will surely help you with the test suite run. > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867263#comment-17867263 ] ASF GitHub Bot commented on HADOOP-18542: - CLevasseur commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2238763477 Hey @anujmodi2021, thanks for the renewed interested in this PR. We have patched a fork on our side since this it was created. I have no idea how to run that test suite properly to be honest. I got inconsistent results last time I tried and was really confused as to how to run it correctly. If you have more experience with it, do you mind trying on your side ? > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867211#comment-17867211 ] ASF GitHub Bot commented on HADOOP-18542: - anujmodi2021 commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2238538839 Also, can you rerun the test suite once rebased and tests fixed and share just the aggregated results part from the run result?? Thanks > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867205#comment-17867205
]
ASF GitHub Bot commented on HADOOP-18542:
-
anujmodi2021 commented on code in PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#discussion_r1683911431
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAccountConfiguration.java:
##
@@ -69,7 +72,6 @@ public class TestAccountConfiguration {
private static final List CONFIG_KEYS =
Review Comment:
This needs to be rebased with latest trunk.
We have modified the test to work with different auth types separately, so
you will need to remove this only for MSI related configs not all OAuth types
> Azure Token provider requires tenant and client IDs despite being optional
> --
>
> Key: HADOOP-18542
> URL: https://issues.apache.org/jira/browse/HADOOP-18542
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, hadoop-thirdparty
>Affects Versions: 3.3.2, 3.3.3, 3.3.4
>Reporter: Carl
>Priority: Major
> Labels: pull-request-available
>
> The `AbfsConfiguration` class requires that we provide a tenant and client ID
> when using the `MsiTokenProvider` class to fetch an authentication token. The
> bug is that those fields are not required by the Azure API, which can infer
> those fields when the call is made from an Azure instance.
> The fix is to make tenant and client ID optional when getting an Azure token
> from the Azure Metadata Service.
> A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262]
> The bug was introduced with HADOOP-17725
> ([https://github.com/apache/hadoop/pull/3041/files])
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17867118#comment-17867118 ] ASF GitHub Bot commented on HADOOP-18542: - mrcnc commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-2237657426 I ran into this earlier today and was initially confused b/c [the docs mention that these fields](https://hadoop.apache.org/docs/stable/hadoop-azure/abfs.html#Azure_Managed_Identity) should be optional but my stack trace showed otherwise. I'd also be happy if we can get this reviewed and merged > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698926#comment-17698926
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1463742413
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 52s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 15s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 25s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 18s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 38s | | The patch does not
generate ASF License warnings. |
| | | 94m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4262 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 260bec46b8bc 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6501eaed6daa0888b1180c1dad456d43c3e1927b |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/8/testReport/ |
| Max. process+thread count | 721 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Azure Token provider
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698907#comment-17698907
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1463712853
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 13m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 59s | | trunk passed |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 21s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 50s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 16s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 106m 43s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4262 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux a3980aaecdc6 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fb5fa8339029b667860a3cc52311efa5c8da659b |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/7/testReport/ |
| Max. process+thread count | 555 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Azure Token provider
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698892#comment-17698892
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1463686612
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 34s | | trunk passed |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 15s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 55s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 4s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 102m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4262 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux fe4472a70cad 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b63e263532cbfb9808efb9e6154902b7315f7158 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/6/testReport/ |
| Max. process+thread count | 579 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Azure Token provider
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698882#comment-17698882 ] ASF GitHub Bot commented on HADOOP-18542: - CLevasseur commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1463679104 That test is working when just running from a fresh install, without changing anything to the auth-keys xml file: ```bash mvn test -Dtest=TestAccountConfiguration#testConfigPropNotFound [INFO] Scanning for projects... [...] [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Running org.apache.hadoop.fs.azurebfs.TestAccountConfiguration [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.36 s - in org.apache.hadoop.fs.azurebfs.TestAccountConfiguration [INFO] [INFO] Results: [INFO] [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 9.540 s [INFO] Finished at: 2023-03-10T12:27:13+01:00 [INFO] ``` I am not sure what you are asking for ? The code this test covers is unrelated to my changes, and is already half broken on the upstream trunk branch. Should I just remove that test ? > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697547#comment-17697547 ] ASF GitHub Bot commented on HADOOP-18542: - steveloughran commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1458542906 that test looks like it is failing because it was expecting a failure and now you've made an option optional, it isn't ``` private static final List CONFIG_KEYS = Collections.unmodifiableList(Arrays.asList( FS_AZURE_ACCOUNT_OAUTH_CLIENT_ENDPOINT, FS_AZURE_ACCOUNT_OAUTH_CLIENT_ID, FS_AZURE_ACCOUNT_OAUTH_CLIENT_SECRET)); ``` now, maybe it is working for you because your auth-keys xml file includes different things. why not try commenting them out locally and running the unit tests (not the iTests) and see what the outcome is > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17694451#comment-17694451 ] ASF GitHub Bot commented on HADOOP-18542: - CLevasseur commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1447813175 @steveloughran @saxenapranav can you have a look please ? > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687020#comment-17687020 ] ASF GitHub Bot commented on HADOOP-18542: - CLevasseur commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1425612739 I just reran the whole test suite on a fresh install, and this time everything passed, without changing the code 路 Note that I had switched to an HNS account last time already. The unit test you mention was failing on the implementation for the `ClientCredsTokenProvider`, which I hadn't changed (I only changed the logic for `MsiTokenProvider`), and was also failing when running from the upstream trunk branch. There might have been some build cache still using a broken version, it's kind of a mystery to me to be honest. Here is the output showing, feel free to run it yourself: ``` /hadoop/hadoop-tools/hadoop-azure# dev-support/testrun-scripts/runtests.sh [INFO] Scanning for projects... [INFO] [INFO] Detecting the operating system and CPU architecture [INFO] [INFO] os.detected.name: linux [INFO] os.detected.arch: x86_64 [INFO] os.detected.version: 5.15 [INFO] os.detected.version.major: 5 [INFO] os.detected.version.minor: 15 [INFO] os.detected.release: ubuntu [INFO] os.detected.release.version: 22.04 [INFO] os.detected.release.like.ubuntu: true [INFO] os.detected.release.like.debian: true [INFO] os.detected.classifier: linux-x86_64 [INFO] [INFO] ---< org.apache.hadoop:hadoop-azure >--- [INFO] Building Apache Hadoop Azure support 3.4.0-SNAPSHOT [INFO] [ jar ]- [INFO] [INFO] - > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17686734#comment-17686734 ] ASF GitHub Bot commented on HADOOP-18542: - steveloughran commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1424712722 the test which is a definite regression is ``` ERROR] testConfigPropNotFound(org.apache.hadoop.fs.azurebfs.TestAccountConfiguration) Time elapsed: 0.19 s <<< FAILURE! java.lang.AssertionError: Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider" at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499) at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:384) at org.apache.hadoop.fs.azurebfs.TestAccountConfiguration.testMissingConfigKey(TestAccountConfiguration.java:399) at org.apache.hadoop.fs.azurebfs.TestAccountConfiguration.testConfigPropNotFound(TestAccountConfiguration.java:386) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) ``` ...that is a test which came in with the mandatoryPassword code. you will have to edit CONFIG_KEYS to remove the keys which are now optional. as to the others, I don't think they are related, more that you are testing through a non-HNS a/c and too much of the test code has drifted to being HNS-only. push up a patch for the CONFIG_KEYS change and I will download and test; if it works for me I'll be happy > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684847#comment-17684847 ] ASF GitHub Bot commented on HADOOP-18542: - steveloughran commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1419534481 @pranavsaxena-microsoft thoughts? would you be able to download this PR and test locally? > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684819#comment-17684819
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1419438553
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 46s | | trunk passed |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 35s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| -1 :x: | javadoc | 0m 41s |
[/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/5/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-azure in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| -1 :x: | javadoc | 0m 25s |
[/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/5/artifact/out/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-azure in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 4s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 10s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 109m 24s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4262 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e42f4c9fa1c2 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 50a67fe0a469a8ab50b9519d19ff0810b1ce3c3b |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results |
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684818#comment-17684818
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1419437664
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 26s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 49m 48s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| -1 :x: | javadoc | 0m 42s |
[/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/3/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-azure in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 14s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| -1 :x: | javadoc | 0m 26s |
[/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/3/artifact/out/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-azure in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 15s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 38s | | The patch does not
generate ASF License warnings. |
| | | 111m 33s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4262 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux ba038e9aa362 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 50a67fe0a469a8ab50b9519d19ff0810b1ce3c3b |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results |
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684817#comment-17684817
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1419435352
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 33s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| -1 :x: | javadoc | 0m 41s |
[/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/4/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-azure in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 10s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 18s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 36s | | the patch passed |
| -1 :x: | javadoc | 0m 25s |
[/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/4/artifact/out/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-azure in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 15s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 108m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4262 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 325f9ba8fe4a 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 50a67fe0a469a8ab50b9519d19ff0810b1ce3c3b |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results |
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684816#comment-17684816
]
ASF GitHub Bot commented on HADOOP-18542:
-
hadoop-yetus commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1419430216
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 3s | | trunk passed |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | | trunk passed |
| -1 :x: | javadoc | 0m 35s |
[/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/2/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-azure in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 34s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 27s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| -1 :x: | javadoc | 0m 23s |
[/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/2/artifact/out/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-azure in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 4s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 53s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 113m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4262/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4262 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux ae810286bc1d 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6e6606c3f16770ed0e2ad8035d849cb5eaa2103d |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results |
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684757#comment-17684757 ] ASF GitHub Bot commented on HADOOP-18542: - CLevasseur commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1419271362 I have had a look at IdentityTransformer (we're talking about [those configuration keys](https://github.com/apache/hadoop/blob/c7ec1897c408257ea1f157123c0d1604b632c048/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/IdentityTransformer.java#L61-L64), right ?)but I am not sure what those fields are for and how to set them, I have tried the following snippet but it failed with similar errors. ```xml fs.azure.identity.transformer.domain.name $superuser fs.azure.identity.transformer.service.principal.substitution.list * ``` I have added a unit-test to check that we can now instantiate an MsiTokenProvider without setting the client and tenant IDs in the configuration. The integration tests that are failing look unrelated to my changes (see test run in my previous comment). > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683544#comment-17683544 ] ASF GitHub Bot commented on HADOOP-18542: - steveloughran commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1414233381 that test_120_terasort(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort failure is happening because yarn wants the owner of a file as determined from getFileStatus to match the yarn user (here: the username of the person running the test) Look at IdentityTransformer and its configuration keys to see what you need to set for a mapping > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681318#comment-17681318 ] ASF GitHub Bot commented on HADOOP-18542: - CLevasseur commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1406479958 Hi, sorry for the delay, it took me a while to set everything up. Unfortunately I am still getting issues when running the integration tests on the `apache/hadoop` repo (i'm not even using my fork at this point). It looks like permission issues, although the roles look correct to me. I followed the doc asking for: ``` 3. A Service principal with RBAC - Storage Blob Data Owner (superuser) role on account 4. A Service principal with no RBAC roles assigned on account. This is needed for checkaccess tests (noRBACUser) 5. A Service principal with Storage Blob Data Contributor RBAC roles assigned on account. This is needed for RBAC tests (contributorRBACUser) 6. A Service principal with Storage Blob Data Reader RBAC roles assigned on account. This is needed for RBAC tests (readerRBACUser) ``` The credentials of each of these 4 account was set in the configuration file. Here is the output of running the HNS-OAuth test suite, can you tell me if those failures are expected, and if not, what I might miss ? ``` [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsStatistics [INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsErrorTranslation [INFO] Running org.apache.hadoop.fs.azurebfs.TestAccountConfiguration [INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsCrc64 [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsPerfTracker [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestExponentialRetryPolicy [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsInputStream [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsRenameRetryRecovery [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.392 s - in org.apache.hadoop.fs.azurebfs.TestAbfsCrc64 [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.225 s - in org.apache.hadoop.fs.azurebfs.TestAbfsErrorTranslation [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestQueryParams [ERROR] Tests run: 9, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.192 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.TestAccountConfiguration [ERROR] testConfigPropNotFound(org.apache.hadoop.fs.azurebfs.TestAccountConfiguration) Time elapsed: 0.19 s <<< FAILURE! java.lang.AssertionError: Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider" at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499) at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:384) at org.apache.hadoop.fs.azurebfs.TestAccountConfiguration.testMissingConfigKey(TestAccountConfiguration.java:399) at org.apache.hadoop.fs.azurebfs.TestAccountConfiguration.testConfigPropNotFound(TestAccountConfiguration.java:386) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641217#comment-17641217 ] ASF GitHub Bot commented on HADOOP-18542: - pranavsaxena-microsoft commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1331933000 Hi, ITestAbfsTerasort.test_110_teragen, ITestAbfsJobThroughManifestCommitter.test_0420_validateJob, ITestAbfsManifestCommitProtocol failures are expected as of now. ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS failure is not expected. As your changes are around OAuth, you would need an HNS account for the testing. Thanks. > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640833#comment-17640833 ] ASF GitHub Bot commented on HADOOP-18542: - CLevasseur commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1330956527 Here is the JIRA to track those integration test failures: https://issues.apache.org/jira/browse/HADOOP-18545 > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[
https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640818#comment-17640818
]
ASF GitHub Bot commented on HADOOP-18542:
-
CLevasseur commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1330911612
The exception for
`ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS` is:
```
[INFO] Running
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess
[ERROR] Tests run: 12, Failures: 1, Errors: 0, Skipped: 10, Time elapsed:
1.521 s <<< FAILURE! - in
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess
[ERROR]
testCheckAccessForAccountWithoutNS(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess)
Time elapsed: 1.15 s <<< FAILURE!
java.lang.AssertionError: Expecting
org.apache.hadoop.security.AccessControlException with text "This request is
not authorized to perform this operation using this permission.", 403 but got :
"void"
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499)
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:529)
at
org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS(ITestAzureBlobFileSystemCheckAccess.java:181)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.lang.Thread.run(Thread.java:829)
```
> are you working with a hierarchical store here? as i think those tests are
failures from that detail, not your change (essentially: etags aren't being
preserved across rename).
No, HNS is disabled on Azure's side, and in the account settings as well:
cat ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml | grep
-B1 -A2 namespace
fs.azure.test.namespace.enabled..dfs.core.windows.net
false
fs.azure.test.namespace.enabled
false
```
> do the same test suite against trunk and see if the tests still fail, if
so, create a new jira (or maybe there is one already). it shouldn't block this
pr
The result is the same, here are the logs I'm getting:
```
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsHttpOperation
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsClient
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsInputStream
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.TestAzureADAuthenticator
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.TestShellDecryptionKeyProvider
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsOutputStream
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAccountConfiguration
[INFO] Running
org.apache.hadoop.fs.azurebfs.services.TestAbfsClientThrottlingAnalyzer
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.976
s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsHttpOperation
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.174
s - in org.apache.hadoop.fs.azurebfs.services.TestShellDecryptionKeyProvider
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsPerfTracker
[ERROR] Tests run: 9, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
2.921 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.TestAccountConfiguration
[ERROR]
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640138#comment-17640138 ] ASF GitHub Bot commented on HADOOP-18542: - steveloughran commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1329440077 ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS looks related...what is the full stack? it may be a different exception came back from the azure call, or it actually worked. other ones are not expected. are you working with a hierarchical store here? as i think those tests are failures from that detail, not your change (essentially: etags aren't being preserved across rename). do the same test suite against trunk and see if the tests still fail, if so, create a new jira (or maybe there is one already). it shouldn't block this pr > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640136#comment-17640136 ] Steve Loughran commented on HADOOP-18542: - what is the stack trace with the error? > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > Labels: pull-request-available > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class to fetch an authentication token. The > bug is that those fields are not required by the Azure API, which can infer > those fields when the call is made from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > ([https://github.com/apache/hadoop/pull/3041/files]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
[ https://issues.apache.org/jira/browse/HADOOP-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640009#comment-17640009 ] ASF GitHub Bot commented on HADOOP-18542: - CLevasseur commented on PR #4262: URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1328997955 I have created this JIRA to track this bug: https://issues.apache.org/jira/browse/HADOOP-18542 > Azure Token provider requires tenant and client IDs despite being optional > -- > > Key: HADOOP-18542 > URL: https://issues.apache.org/jira/browse/HADOOP-18542 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, hadoop-thirdparty >Affects Versions: 3.3.2, 3.3.3, 3.3.4 >Reporter: Carl >Priority: Major > > The `AbfsConfiguration` class requires that we provide a tenant and client ID > when using the `MsiTokenProvider` class. The bug is that those fields are not > required by the Azure API, which can infer those fields when the call is made > from an Azure instance. > The fix is to make tenant and client ID optional when getting an Azure token > from the Azure Metadata Service. > A fix has been submitted here: [https://github.com/apache/hadoop/pull/4262] > The bug was introduced with HADOOP-17725 > (https://github.com/apache/hadoop/pull/3041/files) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
