[ https://issues.apache.org/jira/browse/ARROW-11161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260589#comment-17260589 ]
Antoine Pitrou edited comment on ARROW-11161 at 1/7/21, 3:48 PM: ----------------------------------------------------------------- If you grep for AMZN_XML_CONTENT_TYPE in the C++ AWS SDK you'll see that it's use as a default content-type value for S3 requests. I presume that's the reason for what you're observing, though it's only a guess. was (Author: pitrou): If you grep for AMZN_XML_CONTENT_TYPE you'll see that it's use as a default content-type value for S3 requests. I presume that's the reason for what you're observing, though it's only a guess. > [Python][C++] S3Filesystem: file Content-Type not set correctly? > ---------------------------------------------------------------- > > Key: ARROW-11161 > URL: https://issues.apache.org/jira/browse/ARROW-11161 > Project: Apache Arrow > Issue Type: Bug > Affects Versions: 2.0.0 > Reporter: Nicolas Renkamp > Priority: Major > Labels: filesystem > Attachments: Screen Shot 2021-01-07 at 15.23.07.png, > boto3-metadata.png > > > I am using the Fileystem abstraction to write out html / text files to the > local filesystem as well as s3. > I noticed that when using s3_fs.open_output_stream in combination with > file.write(bytes), the object that gets created has a Content-Type of > 'application/xml' even tough it's plain text, which is problematic for me. > Here is a minimal example: > {code:java} > import boto3 > BUCKET = "my-bucket" > path = f"s3://{BUCKET}/pyarrow_encoding.txt" > s3_fs, output_path = FileSystem.from_uri(path) > with s3_fs.open_output_stream(path=output_path, compression=None) as f: > f.write('hello'.encode('UTF-8')) > s3 = boto3.client('s3') > response = s3.get_object(Bucket=BUCKET, Key='pyarrow_encoding.txt') > print(response['ContentType']) # Output: application/xml > print(response['Body'].read().decode('UTF-8')) # Output: hello > s3.put_object(Bucket=BUCKET, > Key='boto3_encoding.txt', > Body='hello'.encode('UTF-8')) > response = s3.get_object(Bucket=BUCKET, Key='boto3_encoding.txt') > print(response['ContentType']) # Output: binary/octet-stream > print(response['Body'].read().decode('UTF-8')) # Output: hello > {code} > I know, that the S3Filesystem implementation of pyarrow might no have mime > type inference implemented, but I am wondering, why always 'application/xml' > is the resulting Content-Type? Maybe this is hardcoded somewhere? > Originally, I tried this with '.html' files and also there, the objects on s3 > always got the 'application/xml' Content-Type. (Please also see attachment > from the s3 console) > > Any help or pointer is appreciated. > Thank you, > Nicolas -- This message was sent by Atlassian Jira (v8.3.4#803005)