Hi Aaron, I think it is not the responsibility of Flink. Flink uses zookeeper curator to connect the zk server. If multiple zk server are specified, it has an automatic retry mechanism. However, your problem is ip address will change when the EMR instance restarts. Currently, Flink can not support dynamically loading configuration. One quick solution is to use a static ip for EMR master node[1].
Best, Yang [1]. https://aws.amazon.com/premiumsupport/knowledge-center/static-private-ip-master-node-emr/?nc1=h_ls Aaron Langford <aaron.langfor...@gmail.com> 于2020年1月22日周三 上午1:48写道： > Hello Flink Community, > > I'm working on a HA setup of Flink 1.8.1 on AWS EMR and have some > questions about how Flink interacts with Zookeeper when one of the servers > in the quorum specified in flink-conf.yaml goes down and is replaced by a > machine with a new IP address. > > Currently, I configure high-availability.zookeeper.quorum to be the IP > addresses of the 3 master nodes of the EMR cluster, as this is what AWS > does to enable a highly available YARN setup. > > EMR master nodes may go down entirely and need to be replaced by a machine > with a different instance IP address. I will almost certainly need to > perform a rolling configuration update to account for this. But will I need > to restart flink for this to take effect? Is there a way to dynamically > reload these configs when they change? > > Aaron >
