Re: [PR] [FLINK-35232] Add retry settings for GCS connector [flink]
xintongsong closed pull request #24753: [FLINK-35232] Add retry settings for GCS connector URL: https://github.com/apache/flink/pull/24753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-35232] Add retry settings for GCS connector [flink]
xintongsong commented on code in PR #24753:
URL: https://github.com/apache/flink/pull/24753#discussion_r1592308269
##
flink-filesystems/flink-gs-fs-hadoop/src/main/java/org/apache/flink/fs/gs/GSFileSystemOptions.java:
##
@@ -80,6 +84,74 @@ public class GSFileSystemOptions {
.withDescription(
"This option sets the timeout in milliseconds to
read data from an established connection.");
+/**
+ * Flink config option to set the http read timeout. It will be used by
cloud-storage library.
+ */
+public static final ConfigOption GCS_RETRY_MAX_ATTEMPT =
+ConfigOptions.key("gs.retry.max-attempt")
+.intType()
+.defaultValue(6)
Review Comment:
I'm not sure about setting default values for these options. The default
values are ignored anyway because these options are only used in
`getOptional()`, which returns `empty()` if the option is not explicitly
configured by user. We can mention in the description that if not configured,
the GCS default values will be used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-35232] Add retry settings for GCS connector [flink]
flinkbot commented on PR #24753: URL: https://github.com/apache/flink/pull/24753#issuecomment-2088082229 ## CI report: * 1fb740aaac47908e4162fb8cef213c965841288f UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [FLINK-35232] Add retry settings for GCS connector [flink]
JTaky opened a new pull request, #24753: URL: https://github.com/apache/flink/pull/24753 Provide a way to ingest retrier settings configuration to gcs-cloud-storage library in use. ## Brief change log - exposed `gs.retry.max-attempt`, `gs.retry.init-rpc-timeout`, `gs.retry.rpc-timeout-multiplier`, and `gs.retry.rpc-timeout-multiplier` and `gs.retry.total-timeout` configuration. - Pluming retry settings configurations to gcs-cloud-storage client - update the documentation for GCS ## Verifying this change - This change is already covered by existing tests, such as GSRecoverableWriterTest*. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation Does this pull request introduce a new feature? yes If yes, how is the feature documented? docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
