[ https://issues.apache.org/jira/browse/FLINK-13938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973983#comment-16973983 ]
Zili Chen commented on FLINK-13938: ----------------------------------- I send a pull request towards this issue #10187. You can take a look and review it. cc [~fly_in_gis] > Use yarn public distributed cache to speed up containers launch > --------------------------------------------------------------- > > Key: FLINK-13938 > URL: https://issues.apache.org/jira/browse/FLINK-13938 > Project: Flink > Issue Type: New Feature > Components: Deployment / YARN > Reporter: Yang Wang > Assignee: Yang Wang > Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > By default, the LocalResourceVisibility is APPLICATION, so they will be > downloaded only once and shared for all taskmanager containers of a same > application in the same node. However, different applications will have to > download all jars every time, including the flink-dist.jar. I think we could > use the yarn public cache to eliminate the unnecessary jars downloading and > make launching container faster. > > How to use the shared lib feature? > # Upload a copy of flink release binary to hdfs. > # Use the -ysl argument to specify the shared lib > {code:java} > ./bin/flink run -d -m yarn-cluster -p 20 -ysl > hdfs:///flink/release/flink-1.9.0/lib examples/streaming/WindowJoin.jar{code} > > -ysl, --yarnsharedLib <path> Upload a copy of flink lib beforehand > and specify the > path to use public > visibility feature > of YARN NodeManager > localizing > resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)