[
https://issues.apache.org/jira/browse/STORM-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322520#comment-15322520
]
ASF GitHub Bot commented on STORM-1136:
---------------------------------------
Github user abellina commented on a diff in the pull request:
https://github.com/apache/storm/pull/1451#discussion_r66442673
--- Diff: storm-core/src/ui/public/topology.html ---
@@ -384,8 +389,51 @@ <h2 id="topology-resources-header">Topology
resources</h2>
$('#topology-configuration [data-toggle="tooltip"]').tooltip();
$('#topology-actions [data-toggle="tooltip"]').tooltip();
$('#topology-visualization [data-toggle="tooltip"]').tooltip();
+
+ var lagUrl = "/api/v1/topology/"+topologyId+"/lag";
+ $.getJSON(lagUrl,function(lagResponse,status,jqXHR) {
+ if (lagResponse !== null && lagResponse !== undefined &&
lagResponse instanceof Array && lagResponse.length > 0) {
+ var kafkaSpoutsLagTemplate =
$(template).filter("#topology-kafka-spouts-lag-template").html();
+ var spoutsErrorTemplate =
$(template).filter("#topology-spouts-lag-error-template").html();
+
+ var kafkaSpoutLags = lagResponse.filter(function(ele)
{return ele.spoutType === "KAFKA";});
+ var isJson = function (input) {
+ try {
+ JSON.parse(input);
+ } catch (e) {
+ return false;
+ }
+ return true;
+ };
+ var kafkaSpoutsValidResults =
kafkaSpoutLags.filter(function (ele) {return isJson(ele.spoutLagResult);});
+ var kafkaSpoutsErrorResults =
kafkaSpoutLags.filter(function (ele) {return !isJson(ele.spoutLagResult);});
+ var data = {};
+ if (kafkaSpoutsValidResults.length > 0) {
+ data.kafkaSpoutsLagResults = [];
+ kafkaSpoutsValidResults.forEach(function(ele) {
+ var spoutLagResult = JSON.parse(ele.spoutLagResult);
+ spoutLagResult.forEach(function(ele2) {
--- End diff --
@priyank5485 The result currently looks like (if I understand correctly):
[{spoutId: "spout-1", spoutLagResult: "{topic: '..', partition: '..',
logHeadOffset: '..', ...}",
{spoutId: "spout-2", spoutLagResult: "error message"]
I think this should be split into two arrays. Sample proposed response:
// notice no strings, it is strictly JSON.
{valid: [{spoutId: "spout-1", spoutLagResult: [topic: '..', partition:
'..', logHeadOffset: '..', ...]}, ... ],
invalid: [{spoutId: "spout-2", error: ""}, ...]}
This tells you at a glance if something had an error: resp.invalid.length >
0, tells you if there are non-errors spouts: resp.valid.length > 0, and allows
you to loop through resp.valid and resp.invalid to populate your tables.
> Provide a bin script to check consumer lag from KafkaSpout to Kafka topic
> offsets and integrate with storm ui
> -------------------------------------------------------------------------------------------------------------
>
> Key: STORM-1136
> URL: https://issues.apache.org/jira/browse/STORM-1136
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-kafka
> Reporter: Sriharsha Chintalapani
> Assignee: Priyank Shah
>
> We store kafkaspout offsets in zkroot + id path in zookeeper. Kafka provides
> a utility and a protocol request to fetch latest offsets into topic
> {code}
> example:
> bin/kafka-run-classh.sh kafka.tools.GetOffsetTool
> {code}
> we should provide a way for the user to check how far the kafka spout read
> into topic and whats the lag. If we can expose this via UI even better.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
