[jira] [Created] (HADOOP-17636) ABFS: Remove the workaround used for Http PATCH
Bilahari T H created HADOOP-17636: - Summary: ABFS: Remove the workaround used for Http PATCH Key: HADOOP-17636 URL: https://issues.apache.org/jira/browse/HADOOP-17636 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.3.0 Reporter: Bilahari T H JDK7 does not support PATCH, so to to achieve the same as a workaround we use PUT and specify the real method in the X-Http-Method-Override header. This needs to be changed. Since we are using java 8. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17628) ABFS: Distcp contract test testDistCpWithIterator is timing out consistently
Bilahari T H created HADOOP-17628: - Summary: ABFS: Distcp contract test testDistCpWithIterator is timing out consistently Key: HADOOP-17628 URL: https://issues.apache.org/jira/browse/HADOOP-17628 Project: Hadoop Common Issue Type: Sub-task Affects Versions: 3.4.0 Reporter: Bilahari T H The test case testDistCpWithIterator in AbstractContractDistCpTest is consistently timing out. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17615) ADLFS: Update SDK version from 2.3.6 to 2.3.9
Bilahari T H created HADOOP-17615: - Summary: ADLFS: Update SDK version from 2.3.6 to 2.3.9 Key: HADOOP-17615 URL: https://issues.apache.org/jira/browse/HADOOP-17615 Project: Hadoop Common Issue Type: Improvement Components: fs/adl Affects Versions: 3.3.0 Reporter: Bilahari T H Fix For: 3.3.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17536) Suport for customer provided encrption key
Bilahari T H created HADOOP-17536:
-
Summary: Suport for customer provided encrption key
Key: HADOOP-17536
URL: https://issues.apache.org/jira/browse/HADOOP-17536
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H
The data for a particular customer needs to be encrypted on account level. At
server side the APIs will start accepting the encryption key as part of request
headers. The data will be encrypted/decrypted with the given key at the server.
Since the ABFS FileSystem APIs are implementations for Hadoop FileSystem APIs
there is no direct way with which customer can pass the key to ABFS driver. In
this case driver should have the following capabilities so that it can accept
and pass the encryption key as one of the request headers.
# There should be a way to configure the encryption key for different accounts.
# If there is a key specified for a particular account, the same needs to be
sent along with the request headers.
*Config changes*
They key for an account can be specified in the core-site as follows.
fs.azure.account.client-provided-encryption-key.{account
name}.dfs.core.windows.net
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17512) Remove the enable/disable flag for ABFSRemoteListIterator
Bilahari T H created HADOOP-17512: - Summary: Remove the enable/disable flag for ABFSRemoteListIterator Key: HADOOP-17512 URL: https://issues.apache.org/jira/browse/HADOOP-17512 Project: Hadoop Common Issue Type: Improvement Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17503) ABFS: Fix auth failures gettng counted as throttling errors
Bilahari T H created HADOOP-17503: - Summary: ABFS: Fix auth failures gettng counted as throttling errors Key: HADOOP-17503 URL: https://issues.apache.org/jira/browse/HADOOP-17503 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H Currently the throttling intercepter counts the auth failures against throttling errors. This needs to be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17502) IOStatistics collection for listStatusIterator()
Bilahari T H created HADOOP-17502: - Summary: IOStatistics collection for listStatusIterator() Key: HADOOP-17502 URL: https://issues.apache.org/jira/browse/HADOOP-17502 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H Add IOStatistics collection for listStatusIterator -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17475) Implement listStatusIterator
Bilahari T H created HADOOP-17475: - Summary: Implement listStatusIterator Key: HADOOP-17475 URL: https://issues.apache.org/jira/browse/HADOOP-17475 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17474) Optimise abfs incremental listings
Bilahari T H created HADOOP-17474: - Summary: Optimise abfs incremental listings Key: HADOOP-17474 URL: https://issues.apache.org/jira/browse/HADOOP-17474 Project: Hadoop Common Issue Type: New Feature Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17473) Optimise abfs incremental listings
Bilahari T H created HADOOP-17473: - Summary: Optimise abfs incremental listings Key: HADOOP-17473 URL: https://issues.apache.org/jira/browse/HADOOP-17473 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17466) ADL Gen1: Fix the NoClassDefFound error for assertJ.
Bilahari T H created HADOOP-17466: - Summary: ADL Gen1: Fix the NoClassDefFound error for assertJ. Key: HADOOP-17466 URL: https://issues.apache.org/jira/browse/HADOOP-17466 Project: Hadoop Common Issue Type: Test Components: fs/adl Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17459) Fix the test case failures after the contract test update in hadoop-common
Bilahari T H created HADOOP-17459: - Summary: Fix the test case failures after the contract test update in hadoop-common Key: HADOOP-17459 URL: https://issues.apache.org/jira/browse/HADOOP-17459 Project: Hadoop Common Issue Type: Test Components: fs/adl Affects Versions: 3.4.0 Reporter: Bilahari T H Fix For: 3.4.0 Fix the following test case failures which are failing after the contract test update in hadoop-common [ERROR] Failures: [ERROR] TestAdlContractRenameLive>AbstractContractRenameTest.testRenameFileOverExistingFile:131->Assert.fail:88 expected rename(/test/source-256.txt, /test/dest-512.txt) to be rejected with exception, but got false [ERROR] TestAdlContractRenameLive.testRenameFileUnderFile:46 Expecting org.apache.hadoop.security.AccessControlException with text Parent path is not a folder. but got : "void" -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17444) ADLFS: Update SDK version from 2.3.6 to 2.3.9
Bilahari T H created HADOOP-17444: - Summary: ADLFS: Update SDK version from 2.3.6 to 2.3.9 Key: HADOOP-17444 URL: https://issues.apache.org/jira/browse/HADOOP-17444 Project: Hadoop Common Issue Type: Bug Components: fs/adl Affects Versions: 3.4.0 Reporter: Bilahari T H Fix For: 3.4.0 Update SDK version from 2.3.6 to 2.3.9 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17347) ABFS: Read small files completely
Bilahari T H created HADOOP-17347: - Summary: ABFS: Read small files completely Key: HADOOP-17347 URL: https://issues.apache.org/jira/browse/HADOOP-17347 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H Files that are of size smaller than the read buffer size can be considered as small files. In case of such files it would be better to read the full file into the AbfsInputStream buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17289) ABFS: Fixing the testcase ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics
Bilahari T H created HADOOP-17289: - Summary: ABFS: Fixing the testcase ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics Key: HADOOP-17289 URL: https://issues.apache.org/jira/browse/HADOOP-17289 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H The test case is failing when the fs.azure.test.appendblob.enabled=true. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reopened HADOOP-17160: --- The issue is still present. > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Steve Loughran >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17213) ABFS: Test failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics
Bilahari T H created HADOOP-17213: - Summary: ABFS: Test failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics Key: HADOOP-17213 URL: https://issues.apache.org/jira/browse/HADOOP-17213 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H The test ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics fails when the property fs.azure.test.appendblob.enabled is set to true -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17191) ABFS: Run tests with All the auth types
Bilahari T H created HADOOP-17191: - Summary: ABFS: Run tests with All the auth types Key: HADOOP-17191 URL: https://issues.apache.org/jira/browse/HADOOP-17191 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.3.0 Reporter: Bilahari T H Fix For: 3.4.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17183) ABFS: Enable checkaccess API
Bilahari T H created HADOOP-17183: - Summary: ABFS: Enable checkaccess API Key: HADOOP-17183 URL: https://issues.apache.org/jira/browse/HADOOP-17183 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.3.0 Reporter: Bilahari T H Fix For: 3.4.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17166) ABFS: Add configs for maxConcurrentRequestCount and threadpool queue size
Bilahari T H created HADOOP-17166: - Summary: ABFS: Add configs for maxConcurrentRequestCount and threadpool queue size Key: HADOOP-17166 URL: https://issues.apache.org/jira/browse/HADOOP-17166 Project: Hadoop Common Issue Type: Sub-task Affects Versions: 3.3.0 Reporter: Bilahari T H Assignee: Bilahari T H Fix For: 3.4.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17163) ABFS: Add debug log for rename failures
Bilahari T H created HADOOP-17163: - Summary: ABFS: Add debug log for rename failures Key: HADOOP-17163 URL: https://issues.apache.org/jira/browse/HADOOP-17163 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17150) ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests
[ https://issues.apache.org/jira/browse/HADOOP-17150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H resolved HADOOP-17150. --- Resolution: Works for Me Wrong observation > ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests > --- > > Key: HADOOP-17150 > URL: https://issues.apache.org/jira/browse/HADOOP-17150 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Major > Labels: abfsactive > > ITestAzureBlobFileSystemDelegationSAS has tests for the SAS feature in > preview stage. The tests should not run until the API version reflects the > one in preview as when run against production clusters they will fail. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
Bilahari T H created HADOOP-17160: - Summary: ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always Key: HADOOP-17160 URL: https://issues.apache.org/jira/browse/HADOOP-17160 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16740) Backport HADOOP-16612 - "Track Azure Blob File System client-perceived latency" to branch-2
[ https://issues.apache.org/jira/browse/HADOOP-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H resolved HADOOP-16740. --- Resolution: Duplicate Duplicate of [HADOOP-16781|https://issues.apache.org/jira/browse/HADOOP-16781] > Backport HADOOP-16612 - "Track Azure Blob File System client-perceived > latency" to branch-2 > --- > > Key: HADOOP-16740 > URL: https://issues.apache.org/jira/browse/HADOOP-16740 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: abfsactive > > Ref: https://issues.apache.org/jira/browse/HADOOP-16612 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-15761) intermittent failure of TestAbfsClient.validateUserAgent
[
https://issues.apache.org/jira/browse/HADOOP-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilahari T H resolved HADOOP-15761.
---
Resolution: Won't Fix
Please see [HADOOP-16922|https://issues.apache.org/jira/browse/HADOOP-16922]
> intermittent failure of TestAbfsClient.validateUserAgent
>
>
> Key: HADOOP-15761
> URL: https://issues.apache.org/jira/browse/HADOOP-15761
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: HADOOP-15407
> Environment: test suites run from IntelliJ IDEA
>Reporter: Steve Loughran
>Assignee: Bilahari T H
>Priority: Minor
>
> (seemingly intermittent) failure of the pattern matcher in
> {{TestAbfsClient.validateUserAgent}}
> {code}
> java.lang.AssertionError: User agent Azure Blob FS/1.0 (JavaJRE 1.8.0_121;
> MacOSX 10.13.6; openssl-1.0) Partner Service does not match regexp Azure Blob
> FS\/1.0 \(JavaJRE ([^\)]+) SunJSSE-1.8\) Partner Service
> {code}
> Using a regexp is probably too brittle here: safest just to look for some
> specific substring.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16659) ABFS: add missing docs for configuration
[ https://issues.apache.org/jira/browse/HADOOP-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H resolved HADOOP-16659. --- Resolution: Fixed > ABFS: add missing docs for configuration > > > Key: HADOOP-16659 > URL: https://issues.apache.org/jira/browse/HADOOP-16659 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.2.2 >Reporter: Da Zhou >Assignee: Bilahari T H >Priority: Major > > double-check the docs for ABFS and WASB configurations and add the missing > ones. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16913) ABFS: Support for OAuth v2.0 endpoints
[ https://issues.apache.org/jira/browse/HADOOP-16913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H resolved HADOOP-16913. --- Resolution: Fixed > ABFS: Support for OAuth v2.0 endpoints > --- > > Key: HADOOP-16913 > URL: https://issues.apache.org/jira/browse/HADOOP-16913 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > > Driver should upport v2.0 auth endpoints -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17004) ABFS: Improve the ABFS driver documentation
Bilahari T H created HADOOP-17004: - Summary: ABFS: Improve the ABFS driver documentation Key: HADOOP-17004 URL: https://issues.apache.org/jira/browse/HADOOP-17004 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H Fix For: 3.4.0 * Add the missing configuration/settings details * Mention the default vales -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17002) ABFS: Avoid storage calls to check if the account is HNS enabled or not
Bilahari T H created HADOOP-17002: - Summary: ABFS: Avoid storage calls to check if the account is HNS enabled or not Key: HADOOP-17002 URL: https://issues.apache.org/jira/browse/HADOOP-17002 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H Assignee: Bilahari T H Fix For: 3.4.0 Each time an FS instance is created a Getacl call is made. If the call fails with 400 Bad request, the account is determined to be a non-HNS account. Recommendation is to create a config and be able to avoid store calls to determine account HNS status, If config is available, use that to determine account HNS status. If config is not present in core-site, default behaviour will be calling getAcl. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16933) Backport HADOOP-16890- "ABFS: Change in expiry calculation for MSI token provider" to branch-2
Bilahari T H created HADOOP-16933: - Summary: Backport HADOOP-16890- "ABFS: Change in expiry calculation for MSI token provider" to branch-2 Key: HADOOP-16933 URL: https://issues.apache.org/jira/browse/HADOOP-16933 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H Backport "ABFS: Change in expiry calculation for MSI token provider" to branch-2 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16922) ABFS: Change in User-Agent header
Bilahari T H created HADOOP-16922: - Summary: ABFS: Change in User-Agent header Key: HADOOP-16922 URL: https://issues.apache.org/jira/browse/HADOOP-16922 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H Move the configured prefix from the end of the User-Agent value to right after the driver version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16920) ABFS: Make list page size configurable
Bilahari T H created HADOOP-16920: - Summary: ABFS: Make list page size configurable Key: HADOOP-16920 URL: https://issues.apache.org/jira/browse/HADOOP-16920 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H Make list page size configurable -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16915) ABFS: Test failure ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance
Bilahari T H created HADOOP-16915: - Summary: ABFS: Test failure ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance Key: HADOOP-16915 URL: https://issues.apache.org/jira/browse/HADOOP-16915 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H Ref: https://issues.apache.org/jira/browse/HADOOP-16890 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16913) ABFS: Support for OAuth v2.0 endpoints
Bilahari T H created HADOOP-16913: - Summary: ABFS: Support for OAuth v2.0 endpoints Key: HADOOP-16913 URL: https://issues.apache.org/jira/browse/HADOOP-16913 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H Driver should upport v2.0 auth endpoints -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16890) ABFS: Change in exipry calculation for MSI token provider
Bilahari T H created HADOOP-16890: - Summary: ABFS: Change in exipry calculation for MSI token provider Key: HADOOP-16890 URL: https://issues.apache.org/jira/browse/HADOOP-16890 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Reporter: Bilahari T H Assignee: Bilahari T H Set token expiry time as the value of expires_on field from the MSI response in case it is present -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16818) ABFS: Combine append+flush calls for blockblob & appendblob
Bilahari T H created HADOOP-16818: - Summary: ABFS: Combine append+flush calls for blockblob & appendblob Key: HADOOP-16818 URL: https://issues.apache.org/jira/browse/HADOOP-16818 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H Combine append+flush calls for blockblob & appendblob -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16781) Backport HADOOP-16612 "Track Azure Blob File System client-perceived latency" to branch-2
Bilahari T H created HADOOP-16781: - Summary: Backport HADOOP-16612 "Track Azure Blob File System client-perceived latency" to branch-2 Key: HADOOP-16781 URL: https://issues.apache.org/jira/browse/HADOOP-16781 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16740) Backport HADOOP-16612 - "Track Azure Blob File System client-perceived latency" to branch-2
Bilahari T H created HADOOP-16740: - Summary: Backport HADOOP-16612 - "Track Azure Blob File System client-perceived latency" to branch-2 Key: HADOOP-16740 URL: https://issues.apache.org/jira/browse/HADOOP-16740 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.3.0 Reporter: Bilahari T H Fix For: 2.11.0 Ref: https://issues.apache.org/jira/browse/HADOOP-16455 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16734) Backport HADOOP-16455- "ABFS: Implement FileSystem.access() method" to branch-2
Bilahari T H created HADOOP-16734: - Summary: Backport HADOOP-16455- "ABFS: Implement FileSystem.access() method" to branch-2 Key: HADOOP-16734 URL: https://issues.apache.org/jira/browse/HADOOP-16734 Project: Hadoop Common Issue Type: Task Components: fs/azure Affects Versions: 2.0 Reporter: Bilahari T H Assignee: Bilahari T H Fix For: 2.11.0 Backport https://issues.apache.org/jira/browse/HADOOP-16455 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16652) Backport HADOOP-16587 - "Make AAD endpoint configurable on all Auth flows" to branch-2
Bilahari T H created HADOOP-16652: - Summary: Backport HADOOP-16587 - "Make AAD endpoint configurable on all Auth flows" to branch-2 Key: HADOOP-16652 URL: https://issues.apache.org/jira/browse/HADOOP-16652 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Reporter: Bilahari T H Make AAD endpoint configurable on all Auth flows -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16587) Make AAD endpoint configurable on all Auth flows
Bilahari T H created HADOOP-16587: - Summary: Make AAD endpoint configurable on all Auth flows Key: HADOOP-16587 URL: https://issues.apache.org/jira/browse/HADOOP-16587 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Reporter: Bilahari T H Assignee: Bilahari T H Make AAD endpoint configurable on all Auth flows. Currently auth endpoint is hard coded for refreshtoken flow and MSI flow. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16548) ABFS: Config to enable/disable flush operation
Bilahari T H created HADOOP-16548: - Summary: ABFS: Config to enable/disable flush operation Key: HADOOP-16548 URL: https://issues.apache.org/jira/browse/HADOOP-16548 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Reporter: Bilahari T H Assignee: Bilahari T H Make flush operation enabled/disabled through configuration. This is part of performance improvements for ABFS driver. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-16479) ABFS FileStatus.getModificationTime returns localized time instead of UTC
[ https://issues.apache.org/jira/browse/HADOOP-16479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reopened HADOOP-16479: --- > ABFS FileStatus.getModificationTime returns localized time instead of UTC > - > > Key: HADOOP-16479 > URL: https://issues.apache.org/jira/browse/HADOOP-16479 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Joan Sala Reixach >Assignee: Bilahari T H >Priority: Major > Attachments: HADOOP-16479-001.patch, > image-2019-07-31-18-21-53-023.png, image-2019-07-31-18-23-37-349.png > > > As per javadoc, the method FileStatus.getModificationTime() should return the > time in UTC, but it returns the time in the JVM timezone. > The issue origins in AzureBlobFileSystemStore.getFileStatus() itself, since > parseLastModifiedTime() returns a wrong date. I have created a file in Azure > Data Lake Gen2 and when I look at it through the Azure Explorer it shows the > correct modification time, but the method returns -2 hours time (I am in CET > = UTC+2). > Azure Explorer last modified time: > !image-2019-07-31-18-21-53-023.png|width=460,height=45! > AbfsClient parseLastModifiedTime: > !image-2019-07-31-18-23-37-349.png|width=459,height=284! > It shows 15:21 CEST as utcDate, when it should be 15:21 UTC, which results in > the 2 hour loss. > DateFormat.parse uses a localized calendar to parse dates which might be the > source of the issue. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16455) ABFS: Implement the access method
Bilahari T H created HADOOP-16455: - Summary: ABFS: Implement the access method Key: HADOOP-16455 URL: https://issues.apache.org/jira/browse/HADOOP-16455 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Reporter: Bilahari T H Assignee: Bilahari T H Implement the access method -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
