MonitoringInfoSpecs is effectively a list of metrics <https://github.com/apache/beam/blob/c0b60195b17b7cdc46d6ad6548cd41a967e71cde/model/pipeline/src/main/proto/metrics.proto#L60>, but its purpose is to simply define how SDKs should populate MonitoringInfo protos for a RunnerHarness to interpret.
These metrics are provided by the Java and Python SDKs, and Go will soon provide all of them as well. But there is no requirement for a particular runner to support any of these metrics. The DataflowRunner will support these and they metrics are accessible via Dataflow APIs. I am not sure the state of other runners. On Mon, Mar 2, 2020 at 2:47 AM Etienne Chauchot <echauc...@apache.org> wrote: > Hi, > > There is a doc about metrics here: > https://beam.apache.org/documentation/programming-guide/#metrics > > You can also export the metrics to sinks (REST http endpoint and > Graphite), see MetricsOptions class for configuration. > > Still, there is no doc for export on website, I'll add some > > Best > > Etienne > On 28/02/2020 01:07, Pablo Estrada wrote: > > Hi Daniel! > I think +Alex Amato <ajam...@google.com> had tried to have an inventory > of metrics at some point. > Other than that, I don't think we have a document outlining them. > > Can you talk about what you plan to do with them? Do you plan to export > them somehow? Do you plan to add your own? > Best > -P. > > On Thu, Feb 27, 2020 at 11:33 AM Daniel Chen <danx...@gmail.com> wrote: > >> Hi all, >> >> I some questions about the reference to the framework metrics emitted by >> Beam. I would like to leverage these metrics to allow better monitoring of >> by Beam jobs but cannot find any references to the description or a >> complete set of emitted metrics. >> >> Do we have this information documented anywhere? >> >> Thanks, >> Daniel >> >
