Github user Parth-Brahmbhatt commented on the pull request:
https://github.com/apache/storm/pull/354#issuecomment-74308624
@revans2 I have added the nimbus discovery using thrift API to address your
concerns around clients connecting directly to zookeeper.
For fencing around zk, non leader nimbuses still need to write some
information (e.g. they write /storm/code-distributor/ what topologies they have
locally replicated,and now they also write their nimbus summary under
/storm/nimbuses) which means I can not put a single fencing around all zk
write. One way or another I will have to differentiate between leader
operations vs non leader operation. I prefer to put the fencing upfront so any
in memory states are also not altered.
The coupling between storage layer and leader election is a side effect of
having to support a protocol like bittorrent in absence of any highly available
storage from which other hosts could download .torrent files. I can make the
assumption that ICodeDistributor implementation are always backed by highly
available storage systems and that will allow for removal of coupling with
replication count but then unless STORM-411 is done we will have to force users
to use HDFS to achieve nimbus HA.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
