?????? flink use hdfs DistributedCache

2018-09-02 Thread ??????
Rong,thanks for your reply! This is what i need! -- -- ??: "Rong Rong"; : 2018??9??3??(??) 0:02 ??: "??"<244272...@qq.com>; : "user"; : Re: flink use hdfs DistributedCache

Re: flink use hdfs DistributedCache

2018-09-02 Thread Rong Rong
I am not sure if this suits your use case, but Flink YARN cli does support transferring local resource to all YARN nodes. Simply use[1]: bin/flink run -m yarn-cluster -yt or bin/flink run -m yarn-cluster --yarnship should do the trick. It might have not been using the HDFS DistributedCache API t

flink use hdfs DistributedCache

2018-09-02 Thread ??????
hi everyone! can flink submit job which read some custom file distributed by hdfs DistributedCache. like spark can do that with the follow command: bin/spark-submit --master yarn --deploy-mode cluster --files /opt/its007-datacollection-conf.properties#its007-datacollection-conf.properties