[
https://issues.apache.org/jira/browse/HADOOP-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-19104:
------------------------------------
Description:
S3A HeaderProcessing builds up an incomplete list of headers as its mapping of
md to header. entries omits headers including
x-amz-server-side-encryption-aws-kms-key-id
proposed
* review all headers which are stripped from "raw" responses and mapped into
headers
* make sure result of headers matches v1; looks like etags are different
* make sure x-amz-server-side-encryption-aws-kms-key-id gets back
* plus new checksum values
v1 sdk
{code}
# file: s3a://noaa-cors-pds/raw/2024/001/akse/AKSE001x.24_.gz
header.Content-Length="524671"
header.Content-Type="binary/octet-stream"
header.ETag="3e39531220fbd3747d32cf93a79a7a0c"
header.Last-Modified="Tue Jan 02 00:15:13 GMT 2024"
header.x-amz-server-side-encryption="AES256"
{code}
v2 SDK. note how etag is now double quoted.
{code}
# file: s3a://noaa-cors-pds/raw/2024/001/akse/AKSE001x.24_.gz
header.Content-Length="524671"
header.Content-Type="binary/octet-stream"
header.ETag=""3e39531220fbd3747d32cf93a79a7a0c""
header.Last-Modified="Tue Jan 02 00:15:13 GMT 2024"
header.x-amz-server-side-encryption="AES256"
{code}
was:
S3A HeaderProcessing builds up an incomplete list of headers as its mapping of
md to header. entries omits headers including
x-amz-server-side-encryption-aws-kms-key-id
proposed
* review all headers which are stripped from "raw" responses and mapped into
headers
* make sure result of headers matches v1; looks like etags are different
* make sure x-amz-server-side-encryption-aws-kms-key-id gets back
* plus new checksum values
{code}
v1 sdk
{code}
# file: s3a://noaa-cors-pds/raw/2024/001/akse/AKSE001x.24_.gz
header.Content-Length="524671"
header.Content-Type="binary/octet-stream"
header.ETag="3e39531220fbd3747d32cf93a79a7a0c"
header.Last-Modified="Tue Jan 02 00:15:13 GMT 2024"
header.x-amz-server-side-encryption="AES256"
{code}
v2 SDK. note how etag is now double quoted.
{code}
# file: s3a://noaa-cors-pds/raw/2024/001/akse/AKSE001x.24_.gz
header.Content-Length="524671"
header.Content-Type="binary/octet-stream"
header.ETag=""3e39531220fbd3747d32cf93a79a7a0c""
header.Last-Modified="Tue Jan 02 00:15:13 GMT 2024"
header.x-amz-server-side-encryption="AES256"
{code}
> S3A HeaderProcessing to process all metadata entries of HEAD response
> ---------------------------------------------------------------------
>
> Key: HADOOP-19104
> URL: https://issues.apache.org/jira/browse/HADOOP-19104
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Priority: Major
>
> S3A HeaderProcessing builds up an incomplete list of headers as its mapping
> of md to header. entries omits headers including
> x-amz-server-side-encryption-aws-kms-key-id
> proposed
> * review all headers which are stripped from "raw" responses and mapped into
> headers
> * make sure result of headers matches v1; looks like etags are different
> * make sure x-amz-server-side-encryption-aws-kms-key-id gets back
> * plus new checksum values
> v1 sdk
> {code}
> # file: s3a://noaa-cors-pds/raw/2024/001/akse/AKSE001x.24_.gz
> header.Content-Length="524671"
> header.Content-Type="binary/octet-stream"
> header.ETag="3e39531220fbd3747d32cf93a79a7a0c"
> header.Last-Modified="Tue Jan 02 00:15:13 GMT 2024"
> header.x-amz-server-side-encryption="AES256"
> {code}
> v2 SDK. note how etag is now double quoted.
> {code}
> # file: s3a://noaa-cors-pds/raw/2024/001/akse/AKSE001x.24_.gz
> header.Content-Length="524671"
> header.Content-Type="binary/octet-stream"
> header.ETag=""3e39531220fbd3747d32cf93a79a7a0c""
> header.Last-Modified="Tue Jan 02 00:15:13 GMT 2024"
> header.x-amz-server-side-encryption="AES256"
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
