Would you mind sharing your workaround with the community? On Mon, Sep 15, 2014 at 10:17 PM, Bae, Jae Hyeon <metac...@gmail.com> wrote:
> The above pull request didn't work perfectly. After a bunch of testing > experiment, we decided that fixing zkclient itself isn't easy. So we > decided to go with another workaround. > > We're expecting zookeeper 3.5.0 will be stabilized as soon as possible with > the feature to refresh connections and the future version of kafka will > ship that zookeeper version. > > On Mon, Sep 15, 2014 at 8:42 PM, Neha Narkhede <neha.narkh...@gmail.com> > wrote: > > > Thanks for reporting this issue. Agree that this is a problem for Kafka > > users using AWS. Please can you open a JIRA so we can keep track of this? > > > > On Sun, Aug 17, 2014 at 11:41 AM, Bae, Jae Hyeon <metac...@gmail.com> > > wrote: > > > > > Recently, we found the serious ZkClient bug, actual Apache Zookeeper > > client > > > bug, which can bring down broker/consumer on zookeeper push. > > > > > > We're running kafka and zookeeeper in AWS EC2 environment. Zookeeper > > > instances are bound with EIP to give the static hostname for each > > instance, > > > which means even if the EC2 instance is terminated and replaced with > the > > > new one, it will have the same hostname but its private IP bound to the > > > hostname can be changed. > > > > > > The scenario is, if we do rolling push all zookeeper server instances > by > > > terminating and waiting until the new instance joins to the quorum one > by > > > one, finally, ZkClient will try to connect to the old IP addresses > which > > do > > > not exist any more due to DNS caching on Apache Zookeeper client side, > > > please refer to https://issues.apache.org/jira/browse/ZOOKEEPER-338 > > > > > > So, we need to restart kafka brokers and consumers to refresh DNS > cache. > > To > > > solve this problem, I sent the following pull request to ZkClient, > > > https://github.com/sgroschupf/zkclient/pull/26 > > > > > > Please review the above PR. If new version of ZkClient with the > following > > > fix is not released on the schedule of kafka 0.8.2 release, I'd like > > kafka > > > to ship the internally built ZkClient with the fix. I will really > > > appreciate. > > > > > > Thank you > > > Best, Jae > > > > > >
