[jira] [Created] (HADOOP-17636) ABFS: Remove the workaround used for Http PATCH

2021-04-15 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17636:
-

 Summary: ABFS: Remove the workaround used for Http PATCH
 Key: HADOOP-17636
 URL: https://issues.apache.org/jira/browse/HADOOP-17636
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.3.0
Reporter: Bilahari T H


JDK7 does not support PATCH, so to to achieve the same as a workaround we use 
PUT and specify the real method in the X-Http-Method-Override header.
This needs to be changed. Since we are using java 8.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17628) ABFS: Distcp contract test testDistCpWithIterator is timing out consistently

2021-04-08 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17628:
-

 Summary: ABFS: Distcp contract test testDistCpWithIterator is 
timing out consistently 
 Key: HADOOP-17628
 URL: https://issues.apache.org/jira/browse/HADOOP-17628
 Project: Hadoop Common
  Issue Type: Sub-task
Affects Versions: 3.4.0
Reporter: Bilahari T H


The test case testDistCpWithIterator in AbstractContractDistCpTest is 
consistently timing out.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17615) ADLFS: Update SDK version from 2.3.6 to 2.3.9

2021-03-30 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17615:
-

 Summary: ADLFS: Update SDK version from 2.3.6 to 2.3.9
 Key: HADOOP-17615
 URL: https://issues.apache.org/jira/browse/HADOOP-17615
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/adl
Affects Versions: 3.3.0
Reporter: Bilahari T H
 Fix For: 3.3.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17536) Suport for customer provided encrption key

2021-02-18 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17536:
-

 Summary: Suport for customer provided encrption key
 Key: HADOOP-17536
 URL: https://issues.apache.org/jira/browse/HADOOP-17536
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H


The data for a particular customer needs to be encrypted on account level. At 
server side the APIs will start accepting the encryption key as part of request 
headers. The data will be encrypted/decrypted with the given key at the server. 

Since the ABFS FileSystem APIs are implementations for Hadoop FileSystem APIs 
there is no direct way with which customer can pass the key to ABFS driver. In 
this case driver should have the following capabilities so that it can accept 
and pass the encryption key as one of the request headers. 
 # There should be a way to configure the encryption key for different accounts.
 # If there is a key specified for a particular account, the same needs to be 
sent along with the request headers. 

*Config changes* 

They key for an account can be specified in the core-site as follows. 

fs.azure.account.client-provided-encryption-key.{account 
name}.dfs.core.windows.net 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17512) Remove the enable/disable flag for ABFSRemoteListIterator

2021-02-02 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17512:
-

 Summary: Remove the enable/disable flag for ABFSRemoteListIterator
 Key: HADOOP-17512
 URL: https://issues.apache.org/jira/browse/HADOOP-17512
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17503) ABFS: Fix auth failures gettng counted as throttling errors

2021-01-28 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17503:
-

 Summary: ABFS: Fix auth failures gettng counted as throttling 
errors
 Key: HADOOP-17503
 URL: https://issues.apache.org/jira/browse/HADOOP-17503
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H


Currently the throttling intercepter counts the auth failures against 
throttling errors. This needs to be fixed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17502) IOStatistics collection for listStatusIterator()

2021-01-27 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17502:
-

 Summary: IOStatistics collection for listStatusIterator()
 Key: HADOOP-17502
 URL: https://issues.apache.org/jira/browse/HADOOP-17502
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H


Add IOStatistics collection for listStatusIterator



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17475) Implement listStatusIterator

2021-01-17 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17475:
-

 Summary: Implement listStatusIterator
 Key: HADOOP-17475
 URL: https://issues.apache.org/jira/browse/HADOOP-17475
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17474) Optimise abfs incremental listings

2021-01-17 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17474:
-

 Summary: Optimise abfs incremental listings
 Key: HADOOP-17474
 URL: https://issues.apache.org/jira/browse/HADOOP-17474
 Project: Hadoop Common
  Issue Type: New Feature
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17473) Optimise abfs incremental listings

2021-01-17 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17473:
-

 Summary: Optimise abfs incremental listings
 Key: HADOOP-17473
 URL: https://issues.apache.org/jira/browse/HADOOP-17473
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17466) ADL Gen1: Fix the NoClassDefFound error for assertJ.

2021-01-12 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17466:
-

 Summary: ADL Gen1: Fix the NoClassDefFound error for assertJ.
 Key: HADOOP-17466
 URL: https://issues.apache.org/jira/browse/HADOOP-17466
 Project: Hadoop Common
  Issue Type: Test
  Components: fs/adl
Affects Versions: 3.4.0
Reporter: Bilahari T H






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17459) Fix the test case failures after the contract test update in hadoop-common

2021-01-08 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17459:
-

 Summary: Fix the test case failures after the contract test update 
in hadoop-common
 Key: HADOOP-17459
 URL: https://issues.apache.org/jira/browse/HADOOP-17459
 Project: Hadoop Common
  Issue Type: Test
  Components: fs/adl
Affects Versions: 3.4.0
Reporter: Bilahari T H
 Fix For: 3.4.0


Fix the following test case failures which are failing after the contract test 
update in hadoop-common

[ERROR] Failures:
[ERROR] 
TestAdlContractRenameLive>AbstractContractRenameTest.testRenameFileOverExistingFile:131->Assert.fail:88
 expected rename(/test/source-256.txt, /test/dest-512.txt) to be rejected with 
exception, but got false
[ERROR] TestAdlContractRenameLive.testRenameFileUnderFile:46 Expecting 
org.apache.hadoop.security.AccessControlException with text Parent path is not 
a folder. but got : "void"



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17444) ADLFS: Update SDK version from 2.3.6 to 2.3.9

2020-12-22 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17444:
-

 Summary: ADLFS: Update SDK version from 2.3.6 to 2.3.9
 Key: HADOOP-17444
 URL: https://issues.apache.org/jira/browse/HADOOP-17444
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs/adl
Affects Versions: 3.4.0
Reporter: Bilahari T H
 Fix For: 3.4.0


Update SDK version from 2.3.6 to 2.3.9



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17347) ABFS: Read small files completely

2020-11-03 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17347:
-

 Summary: ABFS: Read small files completely
 Key: HADOOP-17347
 URL: https://issues.apache.org/jira/browse/HADOOP-17347
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H


Files that are of size smaller than the read buffer size can be considered as 
small files. In case of such files it would be better to read the full file 
into the AbfsInputStream buffer.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17289) ABFS: Fixing the testcase ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics

2020-09-27 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17289:
-

 Summary: ABFS: Fixing the testcase 
ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics
 Key: HADOOP-17289
 URL: https://issues.apache.org/jira/browse/HADOOP-17289
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H


The test case is failing when the fs.azure.test.appendblob.enabled=true.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Reopened] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always

2020-09-06 Thread Bilahari T H (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bilahari T H reopened HADOOP-17160:
---

The issue is still present.

> ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
> --
>
> Key: HADOOP-17160
> URL: https://issues.apache.org/jira/browse/HADOOP-17160
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Steve Loughran
>Priority: Major
>
> The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out 
> always is timing out always



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17213) ABFS: Test failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics

2020-08-18 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17213:
-

 Summary: ABFS: Test failure 
ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics
 Key: HADOOP-17213
 URL: https://issues.apache.org/jira/browse/HADOOP-17213
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H


The test ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics fails when 
the property fs.azure.test.appendblob.enabled is set to true



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17191) ABFS: Run tests with All the auth types

2020-08-06 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17191:
-

 Summary: ABFS: Run tests with All the auth types
 Key: HADOOP-17191
 URL: https://issues.apache.org/jira/browse/HADOOP-17191
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.3.0
Reporter: Bilahari T H
 Fix For: 3.4.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17183) ABFS: Enable checkaccess API

2020-08-04 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17183:
-

 Summary: ABFS: Enable checkaccess API
 Key: HADOOP-17183
 URL: https://issues.apache.org/jira/browse/HADOOP-17183
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.3.0
Reporter: Bilahari T H
 Fix For: 3.4.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17166) ABFS: Add configs for maxConcurrentRequestCount and threadpool queue size

2020-07-30 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17166:
-

 Summary: ABFS: Add configs for maxConcurrentRequestCount and 
threadpool queue size
 Key: HADOOP-17166
 URL: https://issues.apache.org/jira/browse/HADOOP-17166
 Project: Hadoop Common
  Issue Type: Sub-task
Affects Versions: 3.3.0
Reporter: Bilahari T H
Assignee: Bilahari T H
 Fix For: 3.4.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17163) ABFS: Add debug log for rename failures

2020-07-28 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17163:
-

 Summary: ABFS: Add debug log for rename failures
 Key: HADOOP-17163
 URL: https://issues.apache.org/jira/browse/HADOOP-17163
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Bilahari T H






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Resolved] (HADOOP-17150) ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests

2020-07-28 Thread Bilahari T H (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bilahari T H resolved HADOOP-17150.
---
Resolution: Works for Me

Wrong observation

> ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests
> ---
>
> Key: HADOOP-17150
> URL: https://issues.apache.org/jira/browse/HADOOP-17150
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Sneha Vijayarajan
>Assignee: Sneha Vijayarajan
>Priority: Major
>  Labels: abfsactive
>
> ITestAzureBlobFileSystemDelegationSAS has tests for the SAS feature in 
> preview stage. The tests should not run until the API version reflects the 
> one in preview as when run against production clusters they will fail.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always

2020-07-27 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17160:
-

 Summary: ITestAbfsInputStreamStatistics#testReadAheadCounters 
timing out always
 Key: HADOOP-17160
 URL: https://issues.apache.org/jira/browse/HADOOP-17160
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Bilahari T H


The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always 
is timing out always



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Resolved] (HADOOP-16740) Backport HADOOP-16612 - "Track Azure Blob File System client-perceived latency" to branch-2

2020-07-17 Thread Bilahari T H (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bilahari T H resolved HADOOP-16740.
---
Resolution: Duplicate

Duplicate of [HADOOP-16781|https://issues.apache.org/jira/browse/HADOOP-16781]

> Backport HADOOP-16612 - "Track Azure Blob File System client-perceived 
> latency" to branch-2
> ---
>
> Key: HADOOP-16740
> URL: https://issues.apache.org/jira/browse/HADOOP-16740
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
>  Labels: abfsactive
>
> Ref: https://issues.apache.org/jira/browse/HADOOP-16612



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Resolved] (HADOOP-15761) intermittent failure of TestAbfsClient.validateUserAgent

2020-07-07 Thread Bilahari T H (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bilahari T H resolved HADOOP-15761.
---
Resolution: Won't Fix

Please see [HADOOP-16922|https://issues.apache.org/jira/browse/HADOOP-16922]

> intermittent failure of TestAbfsClient.validateUserAgent
> 
>
> Key: HADOOP-15761
> URL: https://issues.apache.org/jira/browse/HADOOP-15761
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure, test
>Affects Versions: HADOOP-15407
> Environment: test suites run from IntelliJ IDEA
>Reporter: Steve Loughran
>Assignee: Bilahari T H
>Priority: Minor
>
> (seemingly intermittent) failure of the pattern matcher in 
> {{TestAbfsClient.validateUserAgent}}
> {code}
> java.lang.AssertionError: User agent Azure Blob FS/1.0 (JavaJRE 1.8.0_121; 
> MacOSX 10.13.6; openssl-1.0) Partner Service does not match regexp Azure Blob 
> FS\/1.0 \(JavaJRE ([^\)]+) SunJSSE-1.8\) Partner Service
> {code}
> Using a regexp is probably too brittle here: safest just to look for some 
> specific substring.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Resolved] (HADOOP-16659) ABFS: add missing docs for configuration

2020-06-24 Thread Bilahari T H (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bilahari T H resolved HADOOP-16659.
---
Resolution: Fixed

> ABFS: add missing docs for configuration
> 
>
> Key: HADOOP-16659
> URL: https://issues.apache.org/jira/browse/HADOOP-16659
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure
>Affects Versions: 3.2.2
>Reporter: Da Zhou
>Assignee: Bilahari T H
>Priority: Major
>
> double-check the docs for ABFS and WASB configurations and add the missing 
> ones.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Resolved] (HADOOP-16913) ABFS: Support for OAuth v2.0 endpoints

2020-06-24 Thread Bilahari T H (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-16913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bilahari T H resolved HADOOP-16913.
---
Resolution: Fixed

> ABFS: Support for OAuth  v2.0 endpoints
> ---
>
> Key: HADOOP-16913
> URL: https://issues.apache.org/jira/browse/HADOOP-16913
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Major
>
> Driver should upport v2.0 auth endpoints



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17004) ABFS: Improve the ABFS driver documentation

2020-04-20 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17004:
-

 Summary: ABFS: Improve the ABFS driver documentation
 Key: HADOOP-17004
 URL: https://issues.apache.org/jira/browse/HADOOP-17004
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H
 Fix For: 3.4.0


* Add the missing configuration/settings details
* Mention the default vales



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17002) ABFS: Avoid storage calls to check if the account is HNS enabled or not

2020-04-20 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-17002:
-

 Summary: ABFS: Avoid storage calls to check if the account is HNS 
enabled or not
 Key: HADOOP-17002
 URL: https://issues.apache.org/jira/browse/HADOOP-17002
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H
Assignee: Bilahari T H
 Fix For: 3.4.0


Each time an FS instance is created a Getacl call is made. If the call fails 
with 400 Bad request, the account is determined to be a non-HNS account. 

Recommendation is to create a config and be able to avoid store calls to 
determine account HNS status,

If config is available, use that to determine account HNS status. If config is 
not present in core-site, default behaviour will be calling getAcl. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16933) Backport HADOOP-16890- "ABFS: Change in expiry calculation for MSI token provider" to branch-2

2020-03-23 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16933:
-

 Summary: Backport HADOOP-16890- "ABFS: Change in expiry 
calculation for MSI token provider" to branch-2
 Key: HADOOP-16933
 URL: https://issues.apache.org/jira/browse/HADOOP-16933
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Bilahari T H


Backport "ABFS: Change in expiry calculation for MSI token provider" to branch-2



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16922) ABFS: Change in User-Agent header

2020-03-12 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16922:
-

 Summary: ABFS: Change in User-Agent header
 Key: HADOOP-16922
 URL: https://issues.apache.org/jira/browse/HADOOP-16922
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Bilahari T H


Move the configured prefix from the end of the User-Agent value to right after 
the driver version.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16920) ABFS: Make list page size configurable

2020-03-11 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16920:
-

 Summary: ABFS: Make list page size configurable
 Key: HADOOP-16920
 URL: https://issues.apache.org/jira/browse/HADOOP-16920
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Bilahari T H


Make list page size configurable



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16915) ABFS: Test failure ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance

2020-03-08 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16915:
-

 Summary: ABFS: Test failure 
ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance
 Key: HADOOP-16915
 URL: https://issues.apache.org/jira/browse/HADOOP-16915
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Bilahari T H


Ref: https://issues.apache.org/jira/browse/HADOOP-16890



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16913) ABFS: Support for OAuth v2.0 endpoints

2020-03-08 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16913:
-

 Summary: ABFS: Support for OAuth  v2.0 endpoints
 Key: HADOOP-16913
 URL: https://issues.apache.org/jira/browse/HADOOP-16913
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Bilahari T H


Driver should upport v2.0 auth endpoints



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16890) ABFS: Change in exipry calculation for MSI token provider

2020-02-27 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16890:
-

 Summary: ABFS: Change in exipry calculation for MSI token provider
 Key: HADOOP-16890
 URL: https://issues.apache.org/jira/browse/HADOOP-16890
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Reporter: Bilahari T H
Assignee: Bilahari T H


Set token expiry time as the value of expires_on field from the MSI response in 
case it is present



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16818) ABFS: Combine append+flush calls for blockblob & appendblob

2020-01-20 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16818:
-

 Summary: ABFS:  Combine append+flush calls for blockblob & 
appendblob
 Key: HADOOP-16818
 URL: https://issues.apache.org/jira/browse/HADOOP-16818
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Bilahari T H


Combine append+flush calls for blockblob & appendblob



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16781) Backport HADOOP-16612 "Track Azure Blob File System client-perceived latency" to branch-2

2019-12-30 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16781:
-

 Summary: Backport HADOOP-16612 "Track Azure Blob File System 
client-perceived latency" to branch-2
 Key: HADOOP-16781
 URL: https://issues.apache.org/jira/browse/HADOOP-16781
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Bilahari T H






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16740) Backport HADOOP-16612 - "Track Azure Blob File System client-perceived latency" to branch-2

2019-12-03 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16740:
-

 Summary: Backport HADOOP-16612 - "Track Azure Blob File System 
client-perceived latency" to branch-2
 Key: HADOOP-16740
 URL: https://issues.apache.org/jira/browse/HADOOP-16740
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.3.0
Reporter: Bilahari T H
 Fix For: 2.11.0


Ref: https://issues.apache.org/jira/browse/HADOOP-16455



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16734) Backport HADOOP-16455- "ABFS: Implement FileSystem.access() method" to branch-2

2019-11-28 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16734:
-

 Summary: Backport HADOOP-16455- "ABFS: Implement 
FileSystem.access() method" to branch-2
 Key: HADOOP-16734
 URL: https://issues.apache.org/jira/browse/HADOOP-16734
 Project: Hadoop Common
  Issue Type: Task
  Components: fs/azure
Affects Versions: 2.0
Reporter: Bilahari T H
Assignee: Bilahari T H
 Fix For: 2.11.0


Backport https://issues.apache.org/jira/browse/HADOOP-16455



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16652) Backport HADOOP-16587 - "Make AAD endpoint configurable on all Auth flows" to branch-2

2019-10-14 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16652:
-

 Summary: Backport HADOOP-16587 - "Make AAD endpoint configurable 
on all Auth flows" to branch-2
 Key: HADOOP-16652
 URL: https://issues.apache.org/jira/browse/HADOOP-16652
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Reporter: Bilahari T H


Make AAD endpoint configurable on all Auth flows



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16587) Make AAD endpoint configurable on all Auth flows

2019-09-19 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16587:
-

 Summary: Make AAD endpoint configurable on all Auth flows
 Key: HADOOP-16587
 URL: https://issues.apache.org/jira/browse/HADOOP-16587
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Reporter: Bilahari T H
Assignee: Bilahari T H


Make AAD endpoint configurable on all Auth flows. Currently auth endpoint is 
hard coded for refreshtoken flow and MSI flow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16548) ABFS: Config to enable/disable flush operation

2019-09-04 Thread Bilahari T H (Jira)
Bilahari T H created HADOOP-16548:
-

 Summary: ABFS: Config to enable/disable flush operation
 Key: HADOOP-16548
 URL: https://issues.apache.org/jira/browse/HADOOP-16548
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Reporter: Bilahari T H
Assignee: Bilahari T H


Make flush operation enabled/disabled through configuration. This is part of 
performance improvements for ABFS driver.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Reopened] (HADOOP-16479) ABFS FileStatus.getModificationTime returns localized time instead of UTC

2019-08-01 Thread Bilahari T H (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-16479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bilahari T H reopened HADOOP-16479:
---

> ABFS FileStatus.getModificationTime returns localized time instead of UTC
> -
>
> Key: HADOOP-16479
> URL: https://issues.apache.org/jira/browse/HADOOP-16479
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure
>Affects Versions: 3.2.0
>Reporter: Joan Sala Reixach
>Assignee: Bilahari T H
>Priority: Major
> Attachments: HADOOP-16479-001.patch, 
> image-2019-07-31-18-21-53-023.png, image-2019-07-31-18-23-37-349.png
>
>
> As per javadoc, the method FileStatus.getModificationTime() should return the 
> time in UTC, but it returns the time in the JVM timezone.
> The issue origins in AzureBlobFileSystemStore.getFileStatus() itself, since  
> parseLastModifiedTime() returns a wrong date. I have created a file in Azure 
> Data Lake Gen2 and when I look at  it through the Azure Explorer it shows the 
> correct modification time, but the method returns -2 hours time (I am in CET 
> = UTC+2).
> Azure Explorer last modified time:
> !image-2019-07-31-18-21-53-023.png|width=460,height=45!
> AbfsClient parseLastModifiedTime:
> !image-2019-07-31-18-23-37-349.png|width=459,height=284!
> It shows 15:21 CEST as utcDate, when it should be 15:21 UTC, which results in 
> the 2 hour loss.
> DateFormat.parse uses a localized calendar to parse dates which might be the 
> source of the issue.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16455) ABFS: Implement the access method

2019-07-24 Thread Bilahari T H (JIRA)
Bilahari T H created HADOOP-16455:
-

 Summary: ABFS: Implement the access method
 Key: HADOOP-16455
 URL: https://issues.apache.org/jira/browse/HADOOP-16455
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Reporter: Bilahari T H
Assignee: Bilahari T H


Implement the access method



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org