[
https://issues.apache.org/jira/browse/FLINK-35232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas M updated FLINK-35232:
----------------------------
Description:
https://issues.apache.org/jira/browse/FLINK-32877 is tracking ability to
specify transport options in GCS connector. While setting the params enabled
here reduced read timeouts, we still see 503 errors leading to Flink job
restarts.
Thus, in this ticket, we want to specify additional retry settings as noted in
[https://cloud.google.com/storage/docs/retry-strategy#customize-retries.|https://cloud.google.com/storage/docs/retry-strategy#customize-retries]
We want
[these|https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings#methods]
methods available for Flink users so that they can customize their deployment.
was:
https://issues.apache.org/jira/browse/FLINK-32877 is tracking ability to
specify transport options in GCS connector.
In this ticket, we want to specify additional retry settings as noted in
[https://cloud.google.com/storage/docs/retry-strategy#customize-retries.|https://cloud.google.com/storage/docs/retry-strategy#customize-retries]
We want
[these|https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings#methods]
methods available for Flink users so that they can customize their deployment.
The current GCS connector uses the gcs java storage library and bypasses the
hadoop gcs connector which supports multiple http options. There are situations
where GCS takes longer to provide a response for a PUT operation than the
default value.
This change will allow users to customize their connect time and read timeout
based on their application
> Support for retry settings on GCS connector
> -------------------------------------------
>
> Key: FLINK-35232
> URL: https://issues.apache.org/jira/browse/FLINK-35232
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 1.15.3, 1.16.2, 1.17.1
> Reporter: Vikas M
> Assignee: Ravi Singh
> Priority: Major
> Labels: pull-request-available
>
> https://issues.apache.org/jira/browse/FLINK-32877 is tracking ability to
> specify transport options in GCS connector. While setting the params enabled
> here reduced read timeouts, we still see 503 errors leading to Flink job
> restarts.
> Thus, in this ticket, we want to specify additional retry settings as noted
> in
> [https://cloud.google.com/storage/docs/retry-strategy#customize-retries.|https://cloud.google.com/storage/docs/retry-strategy#customize-retries]
> We want
> [these|https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings#methods]
> methods available for Flink users so that they can customize their
> deployment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
