simonelbaz opened a new pull request, #47599:
URL: https://github.com/apache/arrow/pull/47599
…d if ListBucket is not allowed for the user
Thanks for opening a pull request!
### Rationale for this change
This PR gives the user to choose not to create directory in the bucket
before writing dataset.
In case the `create_directory` option is set to FALSE, no verification will
be made by R arrow.
The S3 storage will itself verify if the directory exists and if the users
has the rigth to modify it.
This way no `ListBucket` or ` HeadBucket` are necessary to achieve the
write operation.
```
df |> arrow::write_dataset(
minio$path(paste0("smartsla-bucket/rarrow/")),
partitioning = "qualitative",
create_directory = FALSE,
format = "parquet"
)
```
### What changes are included in this PR?
`create_directory` is now available to the user in the `write_dataset`
function.
Before this PR, this option was automatically set to TRUE (by default).
### Are these changes tested?
Yes
### Are there any user-facing changes?
No, the default value for `create_directory` is still TRUE.
**This PR includes breaking changes to public APIs.** (If there are any
breaking changes to public APIs, please explain which changes are breaking. If
not, you can remove this.)
N/A
**This PR contains a "Critical Fix".** (If the changes fix either (a) a
security vulnerability, (b) a bug that caused incorrect or invalid data to be
produced, or (c) a bug that causes a crash (even when the API contract is
upheld), please provide explanation. If not, you can remove this.)
N/A
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
