an2x commented on issue #27385: URL: https://github.com/apache/beam/issues/27385#issuecomment-1634870291
A few notes: 1) It seems that there was already an [attempt](https://github.com/apache/beam/pull/14765/files#diff-7a37e64a2dd8c51763204b5de1463318e35292177068e8a11ef0cd4764d8b261L415-R411) to reduce the tail latency in the same place, but this change only reduced the max delay from approximately 1000 minutes to 10000 seconds (166 minutes), which is still too much, I think. IMO by default it shouldn't wait for more than 10 minutes, but should wait for more than a few seconds, e.g. something like initial backoff = 1s, exponent = 1.5, max retries = 12 will result in cumulative wait time of 4.5 minutes (and 1.5 minutes before the last attempt). 2) In [other places](https://github.com/apache/beam/blob/bf9473fcada6640377b4e66c87fd22d5d7c1ac3f/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiFinalizeWritesDoFn.java#L107) the RetryManager settings are different and make only 3 retries, though this is probably way too low (3 retries with 1 second initial backoff and 1.5 default backoff exponent means we'll only wait for 1 and 1.5 seconds before the 2nd and 3rd retry). Should we increase the number of retries here? 3) Makes sense to make at least some of these params configurable in `BigQueryIO`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
