Created this SO question: https://stackoverflow.com/questions/48530496/google-dataflow-custom-metrics-not-showing-on-stackdriver
Thanks Andrea! On Tue, Jan 30, 2018 at 9:54 PM Andrea Foegler <foeg...@google.com> wrote: > I would suggest Platform Support or StackOverflow as the best places to > request Dataflow-specific support. > > This could be an issue coordinating between your Stackdriver Account(s) > and your Cloud project(s). We can continue to discuss / investigate > through one of the above forums. > > On Tue, Jan 30, 2018 at 12:30 PM, Carlos Alonso <car...@mrcalonso.com> > wrote: > >> Thanks Andrea!! >> >> Do you mean using the UserVoice forum? >> >> Aside from that, there's something that could be helpful that is that >> when I navigate https://app.google.stackdriver.com/services/dataflow the >> message I get is this: >> "You do not have any resources of this type being monitored by >> Stackdriver." and that's weird as well. As if our Cloud Dataflow wasn't >> properly connected to Stackdriver, but, on the other hand. Some metrics are >> displayed and can be monitored such as System Lag, Watermark, etc... >> >> Thanks! >> >> On Tue, Jan 30, 2018 at 9:20 PM Andrea Foegler <foeg...@google.com> >> wrote: >> >>> Hi Carlos - >>> >>> This sounds like something we should investigate further. Since it >>> appears to be a Dataflow specific question / issue, would you mind posting >>> or following up in a Dataflow-specific forum or through Google Cloud >>> Platform Support: https://cloud.google.com/dataflow/support? Feel free >>> to mention my name in your contact. >>> >>> Cheers >>> Andrea >>> >>> >>> >>> On Tue, Jan 30, 2018 at 10:27 AM, Carlos Alonso <car...@mrcalonso.com> >>> wrote: >>> >>>> Hi Andrea, thank you very much for your response. >>>> >>>> I've followed your directions and only droppedDueToLateness appears. >>>> The way I'm creating those metrics is: >>>> >>>> Metrics.counter("Ingester", "IngestedMessages").inc() >>>> >>>> I can see those counters on the Custom Counters section on the Google >>>> Dataflow UI, but nothing appears on Stackdriver... >>>> >>>> Thanks! >>>> >>>> On Tue, Jan 30, 2018 at 7:22 PM Andrea Foegler <foeg...@google.com> >>>> wrote: >>>> >>>>> Hi Carlos - >>>>> >>>>> Custom metrics can be "listed" by going to the Metric Explorer in >>>>> Stackdriver and entering "custom.googleapis.com/dataflow" in the >>>>> filter. >>>>> If that list contains more than 100 different names, new custom >>>>> metrics will not be created. If this is a case, there should be a message >>>>> in the job log reporting as much. >>>>> (We are working with Stackdriver to improve this experience.) >>>>> >>>>> Also, we do not currently export distribution metrics to Stackdriver >>>>> because we don't yet have a good mechanism to do so. Gauge metrics are >>>>> not >>>>> implemented yet and would not appear in either the Dataflow UI or >>>>> Stackdriver. >>>>> >>>>> These are the only explanations I can think for for these metrics to >>>>> not show up. If neither of these are the case, I'm happy to investigate >>>>> further on a particular instance. >>>>> >>>>> Cheers >>>>> Andrea >>>>> >>>>> >>>>> >>>>> On 2018/01/23 19:59:08, Carlos Alonso <c...@mrcalonso.com> wrote: >>>>> > Hi everyone!!> >>>>> > >>>>> > I'm trying to get a deeper view on my dataflow jobs by measuring >>>>> parts of> >>>>> > it using `Metrics.counter|gauge` but I cannot find how to see them >>>>> on> >>>>> > Stackdriver.> >>>>> > >>>>> > I have a premium Stackdriver account and I can see those counters >>>>> under the> >>>>> > Custom Counters section on the Dataflow UI.> >>>>> > >>>>> > I can see droppedDueToLateness 'custom' counter though on >>>>> Stackdriver that> >>>>> > seems to be created via 'Metrics.counter' as well...> >>>>> > >>>>> > What am I missing?> >>>>> > >>>>> > Regards> >>>>> > >>>>> >>>> >>> >
