[ https://issues.apache.org/jira/browse/FLINK-28643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573852#comment-17573852 ]
Biao Geng commented on FLINK-28643: ----------------------------------- I run into this issue as well when running jobs with large parallism. One possible remedy is to limit the number of the archieved jobs. But a better way may be add limit in flink side so that it would not break down the whole hdfs system. > Job archive file contains too much Job json file on Flink HistoryServer > ------------------------------------------------------------------------ > > Key: FLINK-28643 > URL: https://issues.apache.org/jira/browse/FLINK-28643 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend > Affects Versions: 1.15.1 > Reporter: LI Mingkun > Priority: Major > Attachments: image-2022-07-22-17-04-55-046.png > > > In History Server, 'HistoryServerArchiveFetcher' fetch job archived file,and > pass it into a job dir. > There are more than 1w json files for big parallism job which run out of > inodes > !image-2022-07-22-17-04-55-046.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)