Mokubyow opened a new issue #10634: URL: https://github.com/apache/arrow/issues/10634
I've been having some issues with writing a dataset or table to s3. I'm using pyarrow 4.0.1 with python and am just wondering if I've missing something becuase I keep getting the following error: ``` import pyarrow.dataset as ds from pyarrow import fs import pyarrow.parquet as pq source_uri = "s3://my-existing-bucket/old_prefix/" dataset = ds.dataset(source_uri) target_uri = "s3://my-existing-bucket/new_prefix/" filesystem, path = fs.FileSystem.from_uri(uri) # both of these write methods product the error below # First write type ds.write_dataset(dataset, path, filesystem=filesystem, format="parquet") # Second write type pq.write_to_dataset(dataset.to_table(), path, filesystem=filesystem) ``` **Error** ``` OSError: When creating bucket 'my-existing-bucket': AWS Error [code 100]: Unable to parse ExceptionName: InvalidLocationConstraint Message: The specified location-constraint is not valid ``` I've checked and my region is set to `us-east-1` which is correct for me. Additionally, I was able to write a single parquet file using so I know my credentials are in working order. ``` pq.write_table(dataset.to_table(), path, filesystem=filesystem) ``` Finally, the target bucket was also created in us-east-1 so I don't think it's a region issue, but that is about all I could find online about this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
