My apologies, I ended up resolving this through experimentation. AWS replaces master nodes with the same internal DNS names, so configurations need not be changed.
Aaron On Tue, Jan 21, 2020, 6:33 PM Yang Wang <danrtsey...@gmail.com> wrote: > Hi Aaron, > > I think it is not the responsibility of Flink. Flink uses zookeeper > curator to connect the zk server. If > multiple zk server are specified, it has an automatic retry mechanism. > However, your problem is ip address will change when the EMR instance > restarts. Currently, Flink > can not support dynamically loading configuration. One quick solution is > to use a static ip for EMR > master node[1]. > > > Best, > Yang > > > [1]. > https://aws.amazon.com/premiumsupport/knowledge-center/static-private-ip-master-node-emr/?nc1=h_ls > > Aaron Langford <aaron.langfor...@gmail.com> 于2020年1月22日周三 上午1:48写道： > >> Hello Flink Community, >> >> I'm working on a HA setup of Flink 1.8.1 on AWS EMR and have some >> questions about how Flink interacts with Zookeeper when one of the servers >> in the quorum specified in flink-conf.yaml goes down and is replaced by a >> machine with a new IP address. >> >> Currently, I configure high-availability.zookeeper.quorum to be the IP >> addresses of the 3 master nodes of the EMR cluster, as this is what AWS >> does to enable a highly available YARN setup. >> >> EMR master nodes may go down entirely and need to be replaced by a >> machine with a different instance IP address. I will almost certainly need >> to perform a rolling configuration update to account for this. But will I >> need to restart flink for this to take effect? Is there a way to >> dynamically reload these configs when they change? >> >> Aaron >> >
