Re: Beam Emitted Metrics Reference

2020-03-02 Thread Kenneth Knowles
Also seems like each IO would benefit from an entry in a Transform Catalog with description of any IO-specific metrics it emits. (technically these may not be what you mean by "framework-emitted metrics") Kenn On Mon, Mar 2, 2020 at 9:40 AM Alex Amato wrote: > MonitoringInfoSpecs is effectively

Re: Beam Emitted Metrics Reference

2020-03-02 Thread Alex Amato
MonitoringInfoSpecs is effectively a list of metrics , but its purpose is to simply define how SDKs should populate MonitoringInfo protos for a RunnerHarness to interpret.

Re: Beam Emitted Metrics Reference

2020-03-02 Thread Etienne Chauchot
Hi, There is a doc about metrics here: https://beam.apache.org/documentation/programming-guide/#metrics You can also export the metrics to sinks (REST http endpoint and Graphite), see MetricsOptions class for configuration. Still, there is no doc for export on website, I'll add some Best

Re: Beam Emitted Metrics Reference

2020-02-27 Thread Pablo Estrada
Hi Daniel! I think +Alex Amato had tried to have an inventory of metrics at some point. Other than that, I don't think we have a document outlining them. Can you talk about what you plan to do with them? Do you plan to export them somehow? Do you plan to add your own? Best -P. On Thu, Feb 27, 20

Beam Emitted Metrics Reference

2020-02-27 Thread Daniel Chen
Hi all, I some questions about the reference to the framework metrics emitted by Beam. I would like to leverage these metrics to allow better monitoring of by Beam jobs but cannot find any references to the description or a complete set of emitted metrics. Do we have this information documented a