Hi All,
I'm trying to plot the flink application metrics using grafana backed by
influxdb. I need to plot/monitor the 'numRecordsIn' & 'numRecordsOut' for
each operator/operation. I'm finding it hard to generate the influxdb query
in grafana which can help me make this plot.
I am able to plot the
This works well for me. This will aggregate the data across all sub-task
instances:
SELECT derivative(sum("count"), 1s) FROM "numRecordsIn" WHERE "task_name" =
'Sink: Unnamed' AND $timeFilter GROUP BY time(1s)
You can also plot each sub-task instance separately on the same graph by
doing:
SELECT
Another note. In the example the template variable type is "custom" and
the values have to be enumerated manually. So in your case you would have
to configure all the possible values of "subtask" to be 0-49.
On Tue, Nov 1, 2016 at 2:43 PM, Jamie Grier wrote:
> This works well for me. This will
Ahh.. I haven’t used templating all that much but this also works for your
substask variable so that you don’t have to enumerate all the possible
values:
Template Variable Type: query
query: SHOW TAG VALUES FROM numRecordsIn WITH KEY = "subtask_index"
On Tue, Nov 1, 2016 at 2:51 PM, Jamie Grie
Hi Jamie,
Thank you so much for your response.
The below query:
SELECT derivative(sum("count"), 1s) FROM "numRecordsIn" WHERE "task_name" =
'Sink: Unnamed' AND $timeFilter GROUP BY time(1s)
behaves the same as with the use of the templating variable in the 'All'
case i.e. shows a plots of junk
Hmm. I can't recreate that behavior here. I have seen some issues like
this if you're grouping by a time interval different from the metrics
reporting interval you're using, though. How often are you reporting
metrics to Influx? Are you using the same interval in your Grafana
queries? I see in
I've set the metric reporting frequency to InfluxDB as 10s. In the
screenshot, I'm using Grafana query interval of 1s. I've tried 10s and more
too, the graph shape changes a bit but the incorrect negative values are
still plotted(makes no difference).
Something to add: If the subtasks are less tha
Hi Anchit,
That last bit is very interesting - the fact that it works fine with
subtasks <= 30. It could be that either Influx or Grafana are not able to
keep up with the data being produced. I would guess that the culprit is
Grafana if looking at any particular subtask index works fine and only
Hi Jamie,
Thanks for sharing your thoughts. I'll try and integrate with Graphite to
see if this gets resolved.
Regards,
Anchit
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Metrics-InfluxDB-Grafana-Help-with-query-influxDB-query-for
Hi there,
I am using Graphite and querying it in Grafana is super easy. You just
select fields and they come up automatically for you to select from
depending on how your metric structure in Graphite looks like. You can also
use wildcards.
The only thing I had to do because I am also using containe
10 matches
Mail list logo
