shunping opened a new pull request, #40273:
URL: https://github.com/apache/beam/pull/40273
Follow-up to #40244. With `use_gcsutil_v2 set`, the facade only sent its
GcsPath-typed methods to V2. GcsFileSystem and GcpOptions call the String-typed
and legacy-model methods, so match, list, copy, rename, delete and bucket
creation all stayed on V1.
## Changes
- **GcsUtil:** the legacy methods (`getObject(s)`, `listObjects`, `copy`,
`rename`, `remove`, `create`/`removeBucket`) now delegate to `GcsUtilV2` when
it's enabled. Results are converted back to the JSON API model, so callers
don't change. V1 behavior is kept:
- copy and rename overwrite the destination
- remove ignores a 404
- buckets get projectPrivate ACLs
- **GcsUtilV2:** createBucket takes `BucketTargetOption...` and an optional
project ID. The default upload chunk size now matches gcsio's (8 or 24 MiB,
depending on heap size).
- **ParquetIOLT:** `use_gcsutil_v2` is now sent to Dataflow jobs. It's
joined with the worker experiment, since experiments can only be passed once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
