[Proposal] Defining and Adding SDK Metrics - Looking for Feedback

[Alex Amato]

Hello beam community,

I have put together a proposal
,
and I would like to get some initial feedback to improve upon the ideas
here.

The proposal describes how metrics can be defined and communicated across
the Fn API. Additionally, the proposal describes how to collect custom
metrics without modifying the main beam protos. It also discusses how an
SDK and Runner can be gracefully updated separately to support new metrics.

The Runner must request Metrics from the SDK using a MetricSpec which
describes the metric with a URN string and a corresponding proto. The SDK
can then reply with the populated Metric if it supports the metric's URN
and related protos.

This approach should (hopefully) be general enough to implement all
metrics, including user metrics and allow for graceful updates, user
enabling/disabling metrics collections, etc.

@Etienne
Would you be able to review this as well? I believe that you have also been
looking into Metrics as well and would like to get your feedback.

Proposal (raw link):
https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit

Just an FYI, I will be offline until the 26th, but I wanted to send this
out and get some initial feedback. If there are any urgent questions,
Rafael can respond.

Please take a look and let me know what you think. I look forward to
hearing your feedback
Thanks,
Alex

Re: [Proposal] Defining and Adding SDK Metrics - Looking for Feedback

[Scott Wegner]

Thanks for writing this up! I'll take a look and add comments in the doc.

On Tue, Mar 13, 2018 at 8:55 PM Alex Amato  wrote:

> Hello beam community,
>
> I have put together a proposal
> ,
> and I would like to get some initial feedback to improve upon the ideas
> here.
>
> The proposal describes how metrics can be defined and communicated across
> the Fn API. Additionally, the proposal describes how to collect custom
> metrics without modifying the main beam protos. It also discusses how an
> SDK and Runner can be gracefully updated separately to support new metrics.
>
> The Runner must request Metrics from the SDK using a MetricSpec which
> describes the metric with a URN string and a corresponding proto. The SDK
> can then reply with the populated Metric if it supports the metric's URN
> and related protos.
>
> This approach should (hopefully) be general enough to implement all
> metrics, including user metrics and allow for graceful updates, user
> enabling/disabling metrics collections, etc.
>
> @Etienne
> Would you be able to review this as well? I believe that you have also
> been looking into Metrics as well and would like to get your feedback.
>
> Proposal (raw link):
>
> https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit
>
> Just an FYI, I will be offline until the 26th, but I wanted to send this
> out and get some initial feedback. If there are any urgent questions,
> Rafael can respond.
>
> Please take a look and let me know what you think. I look forward to
> hearing your feedback
> Thanks,
> Alex
>
-- 


Got feedback? http://go/swegner-feedback

Re: [Proposal] Defining and Adding SDK Metrics - Looking for Feedback

[Lukasz Cwik]

Took a look, made some suggestions.


On Mon, Mar 19, 2018 at 11:46 AM Scott Wegner  wrote:

> Thanks for writing this up! I'll take a look and add comments in the doc.
>
> On Tue, Mar 13, 2018 at 8:55 PM Alex Amato  wrote:
>
>> Hello beam community,
>>
>> I have put together a proposal
>> ,
>> and I would like to get some initial feedback to improve upon the ideas
>> here.
>>
>> The proposal describes how metrics can be defined and communicated across
>> the Fn API. Additionally, the proposal describes how to collect custom
>> metrics without modifying the main beam protos. It also discusses how an
>> SDK and Runner can be gracefully updated separately to support new metrics.
>>
>> The Runner must request Metrics from the SDK using a MetricSpec which
>> describes the metric with a URN string and a corresponding proto. The SDK
>> can then reply with the populated Metric if it supports the metric's URN
>> and related protos.
>>
>> This approach should (hopefully) be general enough to implement all
>> metrics, including user metrics and allow for graceful updates, user
>> enabling/disabling metrics collections, etc.
>>
>> @Etienne
>> Would you be able to review this as well? I believe that you have also
>> been looking into Metrics as well and would like to get your feedback.
>>
>> Proposal (raw link):
>>
>> https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit
>>
>> Just an FYI, I will be offline until the 26th, but I wanted to send this
>> out and get some initial feedback. If there are any urgent questions,
>> Rafael can respond.
>>
>> Please take a look and let me know what you think. I look forward to
>> hearing your feedback
>> Thanks,
>> Alex
>>
> --
>
>
> Got feedback? http://go/swegner-feedback
>

Re: [Proposal] Defining and Adding SDK Metrics - Looking for Feedback

[Etienne Chauchot]

Hi Alex,
I left a minor comment as well (but shows as "Anonymous")
Etienne
Le mercredi 14 mars 2018 à 03:47 +, Alex Amato a écrit :
> Hello beam community,
> 
> I have put together a proposal, and I would like to get some initial feedback 
> to improve upon the ideas here.
> 
> The proposal describes how metrics can be defined and communicated across the 
> Fn API. Additionally, the proposal
> describes how to collect custom metrics without modifying the main beam 
> protos. It also discusses how an SDK and
> Runner can be gracefully updated separately to support new metrics.
> 
> The Runner must request Metrics from the SDK using a MetricSpec which 
> describes the metric with a URN string and a
> corresponding proto. The SDK can then reply with the populated Metric if it 
> supports the metric's URN and related
> protos.
> 
> This approach should (hopefully) be general enough to implement all metrics, 
> including user metrics and allow for
> graceful updates, user enabling/disabling metrics collections, etc.
> 
> @Etienne
> Would you be able to review this as well? I believe that you have also been 
> looking into Metrics as well and would
> like to get your feedback.
> 
> Proposal (raw link):
> https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit
> 
> Just an FYI, I will be offline until the 26th, but I wanted to send this out 
> and get some initial feedback. If there
> are any urgent questions, Rafael can respond.
> 
> Please take a look and let me know what you think. I look forward to hearing 
> your feedback
> Thanks,
> Alex