Github user Parth-Brahmbhatt commented on a diff in the pull request:
https://github.com/apache/storm/pull/354#discussion_r22676097
--- Diff: storm-core/src/clj/backtype/storm/cluster.clj ---
@@ -309,6 +324,16 @@
(swap! assignment-version-callback assoc storm-id callback))
(get-version cluster-state (assignment-path storm-id) (not-nil?
callback)))
+ (code-distributor
+ [this callback]
+ (when callback
+ (reset! code-distributor-callback callback))
+ (get-children cluster-state CODE-DISTRIBUTOR-SUBTREE (not-nil?
callback)))
+
+ (code-distributor-info
--- End diff --
@revans2 its not leader-election stuff. This is actually the code
distributor interface as discussed in the design doc which is separate from
leader election. Essentially we have a zookeeper node /storm/code-distributor
which will have one entry for each topology under it and every nimbus that has
code metadata file will be listed under /storm-codedistributor/topologyid/ .
I think the design doc discusses the metafile and code-distributor in more
details but in a nut shell our code distributor protocol relies on a meta file
that each host must download from some other host, most likely the leader
nimbus but not necessary. The meta file has the details about the actual
topology code/conf download location. In case of hdfs it is a directory
location in hdfs , in case of local file system it is a local file system
directory location. This is important because when we start supporting bit
torrent the .torrent file will still need to be distributed. out side of bit
torrent protocol.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
