[jira] [Commented] (HADOOP-14490) Upgrade azure-storage sdk version >5.2.0
[
https://issues.apache.org/jira/browse/HADOOP-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037933#comment-16037933
]
Mingliang Liu commented on HADOOP-14490:
As the related patch is mainly contributed by you, I assign this one to you.
Feel free to un-assign, [~rajesh.balamohan].
> Upgrade azure-storage sdk version >5.2.0
>
>
> Key: HADOOP-14490
> URL: https://issues.apache.org/jira/browse/HADOOP-14490
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Mingliang Liu
>Assignee: Rajesh Balamohan
>
> As required by [HADOOP-14478], we're expecting the {{BlobInputStream}} to
> support advanced {{readFully()}} by taking hints of mark. This can only be
> done by means of sdk version bump.
> cc: [~rajesh.balamohan].
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14490) Upgrade azure-storage sdk version >5.2.0
[
https://issues.apache.org/jira/browse/HADOOP-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16038699#comment-16038699
]
Steve Loughran commented on HADOOP-14490:
-
# patch to update the JAR should be independent of anything else; better
isolation, tracking, etc.
# once the patch is in, change the JIRA name for the exact version.
# And we'll expect a declaration of which endpoint you tested the existing code
against with the updated SDK
What dependencies does this change?
> Upgrade azure-storage sdk version >5.2.0
>
>
> Key: HADOOP-14490
> URL: https://issues.apache.org/jira/browse/HADOOP-14490
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/azure
>Reporter: Mingliang Liu
>Assignee: Rajesh Balamohan
>
> As required by [HADOOP-14478], we're expecting the {{BlobInputStream}} to
> support advanced {{readFully()}} by taking hints of mark. This can only be
> done by means of sdk version bump.
> cc: [~rajesh.balamohan].
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14490) Upgrade azure-storage sdk version >5.2.0
[
https://issues.apache.org/jira/browse/HADOOP-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16045246#comment-16045246
]
Mingliang Liu commented on HADOOP-14490:
>From [HADOOP-14516]:
{quote}
Azure Storage Clients changes between 4.2 and 5.2:
- Fixed Exists() calls on Shares and Directories to now populate metadata. This
was already being done for Files.
- Changed blob constants to support up to 256 MB on put blob for block blobs.
The default value for put blob threshold has also been updated to half of the
maximum, or 128 MB currently.
- Fixed a bug that prevented setting content MD5 to true when creating a new
file.
- Fixed a bug where access conditions, options, and operation context were not
being passed when calling openWriteExisting() on a page blob or a file.
- Fixed a bug where an exception was being thrown on a range get of a blob or
file when the options disableContentMD5Validation is set to false and
useTransactionalContentMD5 is set to true and there is no overall MD5.
- Fixed a bug where retries were happening immediately if a socket exception
was thrown.
- In CloudFileShareProperties, setShareQuota() no longer asserts in bounds.
This check has been moved to create() and uploadProperties() in CloudFileShare.
- Prefix support for listing files and directories.
- Added support for setting public access when creating a blob container
- The public access setting on a blob container is now a container property
returned from downloadProperties.
- Add Message now modifies the PopReceipt, Id, NextVisibleTime, InsertionTime,
and ExpirationTime properties of its CloudQueueMessage parameter.
- Populate content MD5 for range gets on Blobs and Files.
- Added support in Page Blob for incremental copy.
- Added large BlockBlob upload support. Blocks can now support sizes up to 100
MB.
- Added a new, memory-optimized upload strategy for the upload* APIs. This
algorithm only applies for blocks greater than 4MB and when storeBlobContentMD5
and Client-Side Encryption are disabled.
- getQualifiedUri() has been deprecated for Blobs. Please use
getSnapshotQualifiedUri() instead. This new function will return the blob
including the snapshot (if present) and no SAS token.
- getQualifiedStorageUri() has been deprecated for Blobs. Please use
getSnapshotQualifiedStorageUri() instead. This new function will return the
blob including the snapshot (if present) and no SAS token.
- Fixed a bug where copying from a blob that included a SAS token and a
snapshot ommitted the SAS token.
- Fixed a bug in client-side encryption for tables that was preventing the Java
client from decrypting entities encrypted with the .NET client, and vice versa.
- Added support for server-side encryption.
- Added support for getBlobReferenceFromServer methods on CloudBlobContainer to
support retrieving a blob without knowing its type.
- Fixed a bug in the retry policies where 300 status codes were being retried
when they shouldn't be.
{quote}
> Upgrade azure-storage sdk version >5.2.0
>
>
> Key: HADOOP-14490
> URL: https://issues.apache.org/jira/browse/HADOOP-14490
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/azure
>Reporter: Mingliang Liu
>Assignee: Rajesh Balamohan
>
> As required by [HADOOP-14478], we're expecting the {{BlobInputStream}} to
> support advanced {{readFully()}} by taking hints of mark. This can only be
> done by means of sdk version bump.
> cc: [~rajesh.balamohan].
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14490) Upgrade azure-storage sdk version >5.2.0
[
https://issues.apache.org/jira/browse/HADOOP-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051877#comment-16051877
]
Steve Loughran commented on HADOOP-14490:
-
# do we need to switch to non deprecated methods? If so, the patch should
include them.
# are there any new features/perf optimisations needed with this? If so, they
should be covered in followon JIRAs (e.g encryption)
As usual, any patch will need to be accompanied by a declaration of which azure
endpoint was tested against.
> Upgrade azure-storage sdk version >5.2.0
>
>
> Key: HADOOP-14490
> URL: https://issues.apache.org/jira/browse/HADOOP-14490
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/azure
>Reporter: Mingliang Liu
>Assignee: Rajesh Balamohan
>
> As required by [HADOOP-14478], we're expecting the {{BlobInputStream}} to
> support advanced {{readFully()}} by taking hints of mark. This can only be
> done by means of sdk version bump.
> cc: [~rajesh.balamohan].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14490) Upgrade azure-storage sdk version >5.2.0
[
https://issues.apache.org/jira/browse/HADOOP-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064925#comment-16064925
]
Steve Loughran commented on HADOOP-14490:
-
The SDK is updated. So what is this? Is is a duplicate of HADOOP-14536, in
which case: resolve as duplicate, or does it cover actual code changes? In
which case: change title and submit (tested) patches.
> Upgrade azure-storage sdk version >5.2.0
>
>
> Key: HADOOP-14490
> URL: https://issues.apache.org/jira/browse/HADOOP-14490
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Reporter: Mingliang Liu
>Assignee: Rajesh Balamohan
>
> As required by [HADOOP-14478], we're expecting the {{BlobInputStream}} to
> support advanced {{readFully()}} by taking hints of mark. This can only be
> done by means of sdk version bump.
> cc: [~rajesh.balamohan].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14490) Upgrade azure-storage sdk version >5.2.0
[
https://issues.apache.org/jira/browse/HADOOP-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16065136#comment-16065136
]
Mingliang Liu commented on HADOOP-14490:
I think this is not duplicate of [HADOOP-14536] in the sense that what
[HADOOP-14478] depends on in azure sdk is not yet shipped in 5.3.0 version. For
optimizing the seek() performance, we still expect related changes in sdk side,
maybe next version. There are some offline discussion between Rajesh and
Microsoft sdk team, so I think [~rajesh.balamohan] may provide more context.
> Upgrade azure-storage sdk version >5.2.0
>
>
> Key: HADOOP-14490
> URL: https://issues.apache.org/jira/browse/HADOOP-14490
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Reporter: Mingliang Liu
>Assignee: Rajesh Balamohan
>
> As required by [HADOOP-14478], we're expecting the {{BlobInputStream}} to
> support advanced {{readFully()}} by taking hints of mark. This can only be
> done by means of sdk version bump.
> cc: [~rajesh.balamohan].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
