[
https://issues.apache.org/jira/browse/STORM-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15323390#comment-15323390
]
ASF GitHub Bot commented on STORM-1136:
---------------------------------------
Github user abellina commented on a diff in the pull request:
https://github.com/apache/storm/pull/1451#discussion_r66524810
--- Diff: storm-core/src/ui/public/topology.html ---
@@ -384,8 +389,51 @@ <h2 id="topology-resources-header">Topology
resources</h2>
$('#topology-configuration [data-toggle="tooltip"]').tooltip();
$('#topology-actions [data-toggle="tooltip"]').tooltip();
$('#topology-visualization [data-toggle="tooltip"]').tooltip();
+
+ var lagUrl = "/api/v1/topology/"+topologyId+"/lag";
+ $.getJSON(lagUrl,function(lagResponse,status,jqXHR) {
+ if (lagResponse !== null && lagResponse !== undefined &&
lagResponse instanceof Array && lagResponse.length > 0) {
+ var kafkaSpoutsLagTemplate =
$(template).filter("#topology-kafka-spouts-lag-template").html();
+ var spoutsErrorTemplate =
$(template).filter("#topology-spouts-lag-error-template").html();
+
+ var kafkaSpoutLags = lagResponse.filter(function(ele)
{return ele.spoutType === "KAFKA";});
+ var isJson = function (input) {
+ try {
+ JSON.parse(input);
+ } catch (e) {
+ return false;
+ }
+ return true;
+ };
+ var kafkaSpoutsValidResults =
kafkaSpoutLags.filter(function (ele) {return isJson(ele.spoutLagResult);});
+ var kafkaSpoutsErrorResults =
kafkaSpoutLags.filter(function (ele) {return !isJson(ele.spoutLagResult);});
+ var data = {};
+ if (kafkaSpoutsValidResults.length > 0) {
+ data.kafkaSpoutsLagResults = [];
+ kafkaSpoutsValidResults.forEach(function(ele) {
+ var spoutLagResult = JSON.parse(ele.spoutLagResult);
+ spoutLagResult.forEach(function(ele2) {
--- End diff --
@priyank5485 yes, sorry that was my intention but I didn't write it
correctly in my comment, spoutLagResult should be a JSON array. Thanks for the
background!
That said, I think that TopologySpoutLag should do this (my two cents).
That way the output from the rest interface doesn't need further inspection and
some other script can call on your new lag endpoint and get a JSON object they
use without having to do the extra testing. I am not sure that valid/invalid
are the best labels here by the way, just an example I came up with.
> Provide a bin script to check consumer lag from KafkaSpout to Kafka topic
> offsets and integrate with storm ui
> -------------------------------------------------------------------------------------------------------------
>
> Key: STORM-1136
> URL: https://issues.apache.org/jira/browse/STORM-1136
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-kafka
> Reporter: Sriharsha Chintalapani
> Assignee: Priyank Shah
>
> We store kafkaspout offsets in zkroot + id path in zookeeper. Kafka provides
> a utility and a protocol request to fetch latest offsets into topic
> {code}
> example:
> bin/kafka-run-classh.sh kafka.tools.GetOffsetTool
> {code}
> we should provide a way for the user to check how far the kafka spout read
> into topic and whats the lag. If we can expose this via UI even better.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
