[
https://issues.apache.org/jira/browse/STORM-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047903#comment-15047903
]
ASF GitHub Bot commented on STORM-1381:
---------------------------------------
GitHub user Parth-Brahmbhatt opened a pull request:
https://github.com/apache/storm/pull/935
STORM-1381:Adding client side submission hook along with LocalCluster…
… changes to run tets.
Conflicts:
storm-core/src/jvm/backtype/storm/Config.java
Topology hook should work in local mode too.
LocalCluster changes to run nimbus thrift server.
StormSubmitter hook changes along with LocalCluster changes to run tets.
Conflicts:
storm-core/src/clj/backtype/storm/daemon/nimbus.clj
storm-core/src/jvm/backtype/storm/StormSubmitter.java
storm-core/src/jvm/backtype/storm/utils/Utils.java
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Parth-Brahmbhatt/incubator-storm STORM-1381
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/935.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #935
----
commit c5561cfdf520126ad8e5dd8ae39c025eb6041518
Author: Parth Brahmbhatt <[email protected]>
Date: 2015-11-12T20:08:58Z
STORM-1381:Adding client side submission hook along with LocalCluster
changes to run tets.
Conflicts:
storm-core/src/jvm/backtype/storm/Config.java
Topology hook should work in local mode too.
LocalCluster changes to run nimbus thrift server.
StormSubmitter hook changes along with LocalCluster changes to run tets.
Conflicts:
storm-core/src/clj/backtype/storm/daemon/nimbus.clj
storm-core/src/jvm/backtype/storm/StormSubmitter.java
storm-core/src/jvm/backtype/storm/utils/Utils.java
----
> Client side topology submission hook.
> -------------------------------------
>
> Key: STORM-1381
> URL: https://issues.apache.org/jira/browse/STORM-1381
> Project: Apache Storm
> Issue Type: New Feature
> Components: storm-core
> Affects Versions: 0.11.0
> Reporter: Parth Brahmbhatt
> Assignee: Parth Brahmbhatt
> Priority: Trivial
> Fix For: 0.11.0
>
>
> A client side hook is suppose to be invoked when a user submits the topology
> using TopologySubmitter. We already have nimbus side hook for all the
> topology actions however those are good if users don't want to actually
> inspect the topology being submitted or the classes that makes up the
> topology (spouts and bolts) as on nimbus side these classes are not available
> in class path.
> As a concrete example, in hortonworks we wanted to integrate storm with atlas
> to provide complete lineage of data even when it passes through a storm
> topology. Atlas needed to actually look inside the topology components (i.e.
> kafka spout to figure out what topic the data is being pulled from, or hbase
> bolt to figure out which cluster and what table data is being pushed into.)
> to give a meaningful lineage. We originally proposed that they use the server
> side hook but with that they had to download the user uploaded jar and add it
> to the class path dynamically or spin a new jvn whose output will then be
> read by the atlas integration hook.
> The client side hook is suppose to make it easy when the topology itself
> needs to be examined. We are using this in our internal repo for atlas
> integration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
