jorisvandenbossche commented on code in PR #41972:
URL: https://github.com/apache/arrow/pull/41972#discussion_r1627691818
##########
python/pyarrow/_s3fs.pyx:
##########
@@ -236,16 +236,24 @@ cdef class S3FileSystem(FileSystem):
S3FileSystem(proxy_options={'scheme': 'http', 'host': 'localhost',
'port': 8020, 'username': 'username',
'password': 'password'})
- allow_bucket_creation : bool, default False
+ allow_bucket_creation : boolean, default False
Review Comment:
While above it indeed uses `boolean`, doing a quick grep for this pattern in
the full code base, we use `bool` much more often, so I would stick with bool
here
##########
python/pyarrow/_s3fs.pyx:
##########
@@ -236,16 +236,24 @@ cdef class S3FileSystem(FileSystem):
S3FileSystem(proxy_options={'scheme': 'http', 'host': 'localhost',
'port': 8020, 'username': 'username',
'password': 'password'})
- allow_bucket_creation : bool, default False
+ allow_bucket_creation : boolean, default False
Whether to allow CreateDir at the bucket-level. This option may also be
passed in a URI query parameter.
- allow_bucket_deletion : bool, default False
+ allow_bucket_deletion : boolean, default False
Whether to allow DeleteDir at the bucket-level. This option may also be
passed in a URI query parameter.
+ check_directory_existence_before_creation: boolean, default false
Review Comment:
```suggestion
check_directory_existence_before_creation : boolean, default false
```
##########
python/pyarrow/_s3fs.pyx:
##########
@@ -236,16 +236,24 @@ cdef class S3FileSystem(FileSystem):
S3FileSystem(proxy_options={'scheme': 'http', 'host': 'localhost',
'port': 8020, 'username': 'username',
'password': 'password'})
- allow_bucket_creation : bool, default False
+ allow_bucket_creation : boolean, default False
Whether to allow CreateDir at the bucket-level. This option may also be
passed in a URI query parameter.
- allow_bucket_deletion : bool, default False
+ allow_bucket_deletion : boolean, default False
Whether to allow DeleteDir at the bucket-level. This option may also be
passed in a URI query parameter.
+ check_directory_existence_before_creation: boolean, default false
+ Whether to allow pessimistic directory creation in CreateDir function
Review Comment:
It's rather unclear what "pessimistic" means in this context (but I see this
term is used on the C++ side as well). Also, we shouldn't mention "CreateDir"
as that is a C++ function the python user is not familiar with. We can use a
generic term, something like "when creating a directory"
##########
python/pyarrow/_s3fs.pyx:
##########
@@ -236,16 +236,24 @@ cdef class S3FileSystem(FileSystem):
S3FileSystem(proxy_options={'scheme': 'http', 'host': 'localhost',
'port': 8020, 'username': 'username',
'password': 'password'})
- allow_bucket_creation : bool, default False
+ allow_bucket_creation : boolean, default False
Whether to allow CreateDir at the bucket-level. This option may also be
passed in a URI query parameter.
- allow_bucket_deletion : bool, default False
+ allow_bucket_deletion : boolean, default False
Whether to allow DeleteDir at the bucket-level. This option may also be
passed in a URI query parameter.
+ check_directory_existence_before_creation: boolean, default false
+ Whether to allow pessimistic directory creation in CreateDir function
+ if false, then CreateDir will try to create the directory without
checking its
Review Comment:
```suggestion
By default (False), when creating a directory, it will try to create
the directory without checking its
```
I would maybe also include the "It's an optimization to try directory
creation and catch the error, rather than issue two dependent I/O calls." as is
explained in C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
