Thanks to all who have spent their time on this, there were many great suggestions, just another reminder that tomorrow I will be finalizing the documents, unless there are any major objections left. Please take a look at it if you are interested.
I will still welcome feedback at any time :). But I believe we have gathered enough information to produce a good design, which I will start to work on soon. I will begin to build the necessary subset of the new features proposed to support the BigQueryIO metrics use case, proposed. I will likely start with the python SDK first. https://s.apache.org/beam-gcp-debuggability https://s.apache.org/beam-histogram-metrics On Wed, May 13, 2020 at 3:07 PM Alex Amato <ajam...@google.com> wrote: > Thanks again for more feedback :). I have iterated on things again. I'll > report back at the end of the week. If there are no major disagreements > still, I'll close the discussion, believe it to be in a good enough state > to start some implementation. But welcome feedback. > > Latest changes are changing the exponential format to allow denser > buckets. Using only two MonitoringInfoSpec now for all of the IOs to use. > Requiring some labels, but allowing optional > ones for specific IOs to provide more contents. > > https://s.apache.org/beam-gcp-debuggability > https://s.apache.org/beam-histogram-metrics > > On Mon, May 11, 2020 at 4:24 PM Alex Amato <ajam...@google.com> wrote: > >> Thanks for the great feedback so far :). I've included many new ideas, >> and made some revisions. Both docs have changed a fair bit since the >> initial mail out. >> >> https://s.apache.org/beam-gcp-debuggability >> https://s.apache.org/beam-histogram-metrics >> >> PTAL and let me know what you think, and hopefully we can resolve major >> issues by the end of the week. I'll try to finalize things by then, but of >> course always stay open to your great ideas. :) >> >> On Wed, May 6, 2020 at 6:19 PM Alex Amato <ajam...@google.com> wrote: >> >>> Thanks everyone so far for taking a look so far :). >>> >>> I am hoping to have this finalize the two reviews by the end of next >>> week, May 15th. >>> >>> I'll continue to follow up on feedback and make changes, and I will add >>> some more mentions to the documents to draw attention >>> >>> https://s.apache.org/beam-gcp-debuggability >>> https://s.apache.org/beam-histogram-metrics >>> >>> On Wed, May 6, 2020 at 10:00 AM Luke Cwik <lc...@google.com> wrote: >>> >>>> Thanks, also took a look and left some comments. >>>> >>>> On Tue, May 5, 2020 at 6:24 PM Alex Amato <ajam...@google.com> wrote: >>>> >>>>> Hello, >>>>> >>>>> I created another design document. This time for GCP IO Debuggability >>>>> Metrics. Which defines some new metrics to collect in the GCP IO >>>>> libraries. >>>>> This is for monitoring request counts and request latencies. >>>>> >>>>> Please take a look and let me know what you think: >>>>> https://s.apache.org/beam-gcp-debuggability >>>>> >>>>> I also sent out a separate design yesterday ( >>>>> https://s.apache.org/beam-histogram-metrics) which is related as this >>>>> document uses a Histogram style metric :). >>>>> >>>>> I would love some feedback to make this feature the best possible :D, >>>>> Alex >>>>> >>>>
