[ https://issues.apache.org/jira/browse/FLINK-32878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martijn Visser closed FLINK-32878. ---------------------------------- Fix Version/s: 1.19.0 Resolution: Fixed Fixed in apache/flink:master dc1db12137ecad921ae90969d7bfbf1ee7d3d2ef > Add entropy to gcs path for better scalability > ---------------------------------------------- > > Key: FLINK-32878 > URL: https://issues.apache.org/jira/browse/FLINK-32878 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem > Affects Versions: 1.15.4, 1.16.2, 1.17.1 > Reporter: Jayadeep Jayaraman > Assignee: Cheena > Priority: Major > Labels: pull-request-available > Fix For: 1.19.0 > > > Currently GCS is used as a backend for both checkpointing and sink. In both > these cases the file names are sequential which causes hotspotting in GCS and > results in HTTP 5XX status code. > > As per [GCS best > practices]([https://cloud.google.com/storage/docs/request-rate)] it is > advisable to spread out the object creation with random names close to the > beginning of the file name. > > There is similar work done already for S3 as part of FLINK-9061 -- This message was sent by Atlassian Jira (v8.20.10#820010)