westonpace commented on code in PR #12701:
URL: https://github.com/apache/arrow/pull/12701#discussion_r854532343
##########
python/pyarrow/dataset.py:
##########
@@ -852,6 +852,8 @@ def file_visitor(written_file):
dataset. The first time each partition directory is encountered
the entire directory will be deleted. This allows you to overwrite
old partitions completely.
+ create_dir : bool, default True
+ Flag to restrict or allow creating directory while writing a dataset.
Review Comment:
```suggestion
create_dir : bool, default True
If False, directories will not be created. This can be useful for
filesystems
that do not require directories.
```
##########
cpp/src/arrow/dataset/file_base.h:
##########
@@ -387,6 +387,10 @@ struct ARROW_DS_EXPORT FileSystemDatasetWriteOptions {
/// Controls what happens if an output directory already exists.
ExistingDataBehavior existing_data_behavior = ExistingDataBehavior::kError;
+ /// Whether to attempt creating the dataset directory.
+ /// This can be set to false to work around limited permissions on some
filesystems.
+ bool create_dir = true;
Review Comment:
```suggestion
/// \brief If false the dataset writer will not create directories
/// This is mainly intended for filesystems that do not require
directories such as S3
bool create_dir = true;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
