[
https://issues.apache.org/jira/browse/ARROW-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17456620#comment-17456620
]
Luis Morales edited comment on ARROW-14930 at 12/9/21, 6:02 PM:
----------------------------------------------------------------
those folders are for scality. it uses it own internal structure to store both
the data and the metadata. it's the S3 layer that translates that structures to
S3-like structure of buckets and files...
in my use case i create the buckets and store the parquets using s3cmd.
was (Author: JIRAUSER281000):
those folders are for scality. it uses it own internal structure to store both
the data and the metadata. it's the S3 layer that translates that structures to
S3-like structure of buckets and files...
> [C++][Python] FileNotFound with Scality accessed through S3 APIs
> ----------------------------------------------------------------
>
> Key: ARROW-14930
> URL: https://issues.apache.org/jira/browse/ARROW-14930
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
> Affects Versions: 6.0.1
> Environment: linux + python 3.8
> Reporter: Luis Morales
> Priority: Major
> Labels: s3
> Fix For: 6.0.2
>
>
> When using dataset.Dataset with S3FileSystem with compatible S3 object
> sotrage, get an FileNotFoundError.
>
> My code:
>
> scality = fs.S3FileSystem(access_key='accessKey1',
> secret_key='verySecretKey1', endpoint_override="http://localhost:8000";,
> region="")
> data = ds.dataset("dasynth/parquet/taxies/2019_june/", format="parquet",
> partitioning="hive", filesystem=scality)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
