Hello beam community, I have put together a proposal <https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit>, and I would like to get some initial feedback to improve upon the ideas here.
The proposal describes how metrics can be defined and communicated across the Fn API. Additionally, the proposal describes how to collect custom metrics without modifying the main beam protos. It also discusses how an SDK and Runner can be gracefully updated separately to support new metrics. The Runner must request Metrics from the SDK using a MetricSpec which describes the metric with a URN string and a corresponding proto. The SDK can then reply with the populated Metric if it supports the metric's URN and related protos. This approach should (hopefully) be general enough to implement all metrics, including user metrics and allow for graceful updates, user enabling/disabling metrics collections, etc. @Etienne Would you be able to review this as well? I believe that you have also been looking into Metrics as well and would like to get your feedback. Proposal (raw link): https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit Just an FYI, I will be offline until the 26th, but I wanted to send this out and get some initial feedback. If there are any urgent questions, Rafael can respond. Please take a look and let me know what you think. I look forward to hearing your feedback Thanks, Alex
