amoeba commented on code in PR #41998:
URL: https://github.com/apache/arrow/pull/41998#discussion_r1699394578
##########
r/tests/testthat/test-s3-minio.R:
##########
@@ -55,6 +56,7 @@ limited_fs <- S3FileSystem$create(
endpoint_override = paste0("localhost:", minio_port),
allow_bucket_creation = FALSE,
allow_bucket_deletion = FALSE
Review Comment:
```suggestion
allow_bucket_deletion = FALSE,
```
##########
r/R/filesystem.R:
##########
@@ -156,6 +156,13 @@ FileSelector$create <- function(base_dir, allow_not_found
= FALSE, recursive = F
#' buckets if `$CreateDir()` is called on the bucket level (default
`FALSE`).
#' - `allow_bucket_deletion`: logical, if TRUE, the filesystem will delete
#' buckets if`$DeleteDir()` is called on the bucket level (default `FALSE`).
+#' - `check_directory_existence_before_creation`: logical, if FALSE, when
creating a directory the code will
+#' . not check if it already exists or not. It's an optimization to try
directory creation and catch the error,
+#' rather than issue two dependent I/O calls.
+#' if TRUE, when creating a directory the code will only create the
directory when necessary
Review Comment:
```suggestion
#' if `TRUE`, when creating a directory the code will only create the
directory when necessary
```
##########
r/NEWS.md:
##########
@@ -18,6 +18,7 @@
-->
# arrow 16.1.0.9000
+* Expose an option `check_directory_existence_before_creation` in
`S3FileSystem` which defaults to false. If it's set to false, when creating a
directory the code will not check if it already exists or not. It's an
optimization to try directory creation and catch the error, rather than issue
two dependent I/O calls. If true, when creating a directory the code will only
create the directory when necessary at the cost of extra I/O calls. This can be
used for key/value cloud storage which has a hard rate limit to number of
object mutation operations or scenerios such as the directories already exist
and you do not have creation access.
Review Comment:
```suggestion
* Expose an option `check_directory_existence_before_creation` in
`S3FileSystem` which defaults to `FALSE`. If it's set to false, when creating a
directory the code will not check if it already exists or not. It's an
optimization to try directory creation and catch the error, rather than issue
two dependent I/O calls. If set to `TRUE`, when creating a directory the code
will only create the directory when necessary at the cost of extra I/O calls.
This can be used for key/value cloud storage which has a hard rate limit to
number of object mutation operations or scenarios such as the directories
already exist and you do not have creation access.
```
##########
r/R/filesystem.R:
##########
@@ -156,6 +156,13 @@ FileSelector$create <- function(base_dir, allow_not_found
= FALSE, recursive = F
#' buckets if `$CreateDir()` is called on the bucket level (default
`FALSE`).
#' - `allow_bucket_deletion`: logical, if TRUE, the filesystem will delete
#' buckets if`$DeleteDir()` is called on the bucket level (default `FALSE`).
+#' - `check_directory_existence_before_creation`: logical, if FALSE, when
creating a directory the code will
Review Comment:
```suggestion
#' - `check_directory_existence_before_creation`: logical, if `FALSE`, when
creating a directory the code will
```
##########
r/R/filesystem.R:
##########
@@ -156,6 +156,13 @@ FileSelector$create <- function(base_dir, allow_not_found
= FALSE, recursive = F
#' buckets if `$CreateDir()` is called on the bucket level (default
`FALSE`).
#' - `allow_bucket_deletion`: logical, if TRUE, the filesystem will delete
#' buckets if`$DeleteDir()` is called on the bucket level (default `FALSE`).
+#' - `check_directory_existence_before_creation`: logical, if FALSE, when
creating a directory the code will
+#' . not check if it already exists or not. It's an optimization to try
directory creation and catch the error,
+#' rather than issue two dependent I/O calls.
+#' if TRUE, when creating a directory the code will only create the
directory when necessary
+#' at the cost of extra I/O calls. This can be used for key/value cloud
storage which has
+#' a hard rate limit to number of object mutation operations or scenerios
such as
Review Comment:
```suggestion
#' a hard rate limit to number of object mutation operations or scenarios
such as
```
##########
r/tests/testthat/test-s3-minio.R:
##########
@@ -55,6 +56,7 @@ limited_fs <- S3FileSystem$create(
endpoint_override = paste0("localhost:", minio_port),
allow_bucket_creation = FALSE,
allow_bucket_deletion = FALSE
+ check_directory_existence_before_creation = false,
Review Comment:
```suggestion
check_directory_existence_before_creation = FALSE,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
