[ https://issues.apache.org/jira/browse/YARN-6426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shilun Fan updated YARN-6426: ----------------------------- Target Version/s: 3.5.0 (was: 3.4.0) > Compress ZK YARN keys to scale up (especially AppStateData > ---------------------------------------------------------- > > Key: YARN-6426 > URL: https://issues.apache.org/jira/browse/YARN-6426 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn > Affects Versions: 3.0.0-alpha2 > Reporter: Roni Burd > Assignee: Roni Burd > Priority: Major > Labels: patch > Attachments: zkcompression.patch > > > ZK today stores the protobuf files uncompressed. This is not an issue except > that if a customer job has thousands of files, AppStateData will store the > user context as a string with multiple URLs and it is easy to get to 1MB or > more. > This can put unnecessary strain on ZK and make the process slow. > The proposal is to simply compress protobufs before sending them to ZK -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org