Hi vino Should the flink CLI have access to the path mentioned in *high-availability.storageDir*? If my flink cluster is on set of machines and i submit my job from flink CLI from another independent machine by giving necessary details will the CLI try to access *high-availability.storageDir *path?
I'm aware of the fact that flink client will connect to zookeeper to get leader address and necessary information for job submission but my confusion is with *high-availability.storageDir* and its necessity in flink CLI configuration. On Mon, Jul 16, 2018 at 2:44 PM, vino yang <yanghua1...@gmail.com> wrote: > Hi Sampath, > > Flink CLI need to retrieve the JobManager leader address, so it need to > access the HA specific configuration. Because if based on Zookeeper to > implement the HA, the leader address information will fetch from Zookeeper. > > The main use of config item *high-availability.storageDir* is storage > (Job graph, checkpoint and so on). Actually, the real data is stored under > this path which used to recover purpose, zookeeper just store a state > handle. > > --- > Thanks. > vino. > > > 2018-07-16 15:28 GMT+08:00 Sampath Bhat <sam414255p...@gmail.com>: > >> >> ---------- Forwarded message ---------- >> From: Sampath Bhat <sam414255p...@gmail.com> >> Date: Fri, Jul 13, 2018 at 3:18 PM >> Subject: Flink CLI properties with HA >> To: user <user@flink.apache.org> >> >> >> Hello >> >> When HA is enabled in the flink cluster and if I've to submit job via >> flink CLI then in the flink-conf.yaml of flink CLI should contain this >> properties - >> high-availability: zookeeper >> high-availability.cluster-id: flink >> high-availability.zookeeper.path.root: flink >> high-availability.storageDir: <some path> >> high-availability.zookeeper.quorum: <zookeeper IP:port> >> >> What is the need of high-availability.storageDir for flink CLI. Does this >> mean that even flink client should be able to access the mentioned path or >> is it some check being done on the property name? >> >> Without these properties flink cli will not be able to submit job to >> flink cluster when HA is enabled. >> >> >
