Thanks for the clarification Hinduja. I agree that this is not following the standard guidelines for Prometheus exporters. But I came across the following in the documentation[1] <https://prometheus.io/docs/instrumenting/writing_exporters/>, which somewhat explains what you have developed.
> In other cases, metrics from the system are completely non-standard, > depending on the usage of the system and the underlying application. In > that case the user has to tell us how to transform the metrics. > We can get an opinion from the Prometheus community using their channels at [2] <https://prometheus.io/community/>. Regarding Telegraf, even I had a look at the online resources and was unable to locate a Java client that can be extended to develop a plugin. But I came across a Prometheus input plugin[3] <https://github.com/influxdata/telegraf/blob/release-1.9/plugins/inputs/prometheus/README.md>, which can work with the metrics that you publish using the Siddhi Prometehus sink. This might not be the best solution performance-wise due to this additional hop(Prometheus input plugin), but this can be used to publish the metrics to Telegraf. So, we can actually use your solution with Telegraf too. [1] https://prometheus.io/docs/instrumenting/writing_exporters/ [2] https://prometheus.io/community/ [3] https://github.com/influxdata/telegraf/blob/release-1.9/plugins/inputs/prometheus/README.md Thanks. Dilhasha On Wed, Dec 5, 2018 at 1:05 PM Hinduja Balasubramaniyam <hind...@wso2.com> wrote: > Hi Dilhasa, > > Thank you for your kind suggestions. > > - According to my current implementation, the sink does not expose any > metrics by default. It exposes only the user defined metrics inside the > Siddhi application. But, if that changes in future, I'll update the read me > file as well. > > > - I need to clarify that the Prometheus sink does not monitor siddhi, > it provides user a chance to create and export metrics at the preferred > endpoint through the sink. Therefore the sink will not be supporting some > of the general rules of a Prometheus exporter[1] > <https://prometheus.io/docs/instrumenting/writing_exporters/>. For > example, the sink enables the user to add custom configurations which is > not supposed to be in an exporter. So, I think we need to reconsider the > idea. > > > - I have read some documentations [2] > <https://github.com/influxdata/telegraf/blob/master/README.md>, [3] > <https://docs.influxdata.com/telegraf/v1.9/> on Telegraf. And I agree > that the functionality and concepts are very similar to Prometheus. But, I > could not find any java client for developing a plugin. So I think, we need > to further study on Telegraf and verify the possibility of having a siddhi > extension for it. > > [1] https://prometheus.io/docs/instrumenting/writing_exporters/ > [2] https://github.com/influxdata/telegraf/blob/master/README.md > [3] https://docs.influxdata.com/telegraf/v1.9/ > > Best regards, > Hinduja. > > On Tue, Dec 4, 2018 at 11:07 AM Fathima Dilhasha <dilha...@wso2.com> > wrote: > >> Hi Hinduja, >> >> I went through the documentaion and implementation. It's a good >> initiative to make the SP analytics available to a widely used monitoring >> tool like Prometheus. >> I have few suggestions on some improvements. >> >> - In the documentation, I did not come across a section where we have >> listed the available metrics. Do we have a set of metrics that will be >> available by default if we use this siddhi sink/exporter? >> If so, it would be good to include it in the read me file of the >> repository. >> - When we have the implementation added to wso2-extensions, we can >> look at getting this sink listed in the third party exporters[1] for >> Prometehus. >> - Do we have plans on implementing a metrics agent for Telegraph as >> well? Since, Telegraph is also another widely used tool, we can look at it >> too if time permits. WDYT? >> >> [1] https://prometheus.io/docs/instrumenting/exporters/ >> >> Thanks. >> Dilhasha >> >> On Sat, Dec 1, 2018 at 7:43 AM Hinduja Balasubramaniyam <hind...@wso2.com> >> wrote: >> >>> Hi all, >>> >>> I have implemented the sink for Siddhi-io-prometheus. The implementation >>> can be found here <https://github.com/HindujaB/siddhi-io-prometheus>. >>> This will be available inside wso2-extensions >>> <https://github.com/wso2-extensions> soon. >>> >>> The code review was held on 13th November 2018. Please find the notes >>> from the following email thread. >>> >>> - Updated invitation: [Code Review] Prometheus sink for Siddhi @ Tue >>> Nov 13, 2018 2:30pm - 3.30pm (IST) (Analytics Group) >>> >>> In addition, the user story review for Prometheus source was held on >>> 30th November 2018. Please find the notes from the following email thread. >>> >>> - Updated invitation: [User Story Review] Prometheus source for >>> Siddhi @ Fri Nov 30, 2018 11am - 12pm (IST) (Analytics Group) >>> >>> The updated user stories for siddhi-io-prometheus can be found here >>> <https://docs.google.com/document/d/1_lCn9hOdYkbG6hnYrqnB1Ww2ds5zMfFa9aJZFQjWY0E/edit?usp=sharing> >>> . >>> Please note that I am currently working on the implementation of >>> Prometheus source for Siddhi. >>> Best regards, >>> >>> On Fri, Oct 12, 2018 at 6:59 PM Hinduja Balasubramaniyam < >>> hind...@wso2.com> wrote: >>> >>>> Hi all, >>>> >>>> >>>> >>>> Prometheus is a system monitoring toolkit that scrapes and stores time >>>> series data. It provides a data model which can be identified by metrics >>>> with metric name, metric type and key/value pairs. Prometheus also supports >>>> retrieval of metrics over HTTP. >>>> >>>> >>>> >>>> My project is to develop a connector to WSO2 SP that supports >>>> Prometheus where the user be able to publish events to metrics and consume >>>> events from the metrics to perform analytics using Siddhi app. >>>> >>>> >>>> >>>> I have tried to instrument Prometheus in a simple java program using >>>> the Prometheus java client. The mentioned java code is available here, >>>> >>>> >>>> >>>> https://github.com/HindujaB/prometheus_trial/blob/master/src/main/java/Prometheus_Test.java >>>> >>>> >>>> Currently I am studying on instrumenting Prometheus in >>>> Siddhi-io-extension. >>>> >>>> >>>> >>>> Regards, >>>> >>>> Hinduja >>>> >>>> -- >>>> *Hinduja Balasubramaniyam* >>>> Software Engineering Intern | WSO2 >>>> >>>> Email : hind...@wso2.com >>>> Mobile : +94766612192 >>>> >>>> >>>> >>> >>> -- >>> *Hinduja Balasubramaniyam* >>> Software Engineering Intern | WSO2 >>> >>> Email : hind...@wso2.com >>> Mobile : +94766612192 >>> >>> >>> >> >> -- >> *Fathima Dilhasha* | Senior Software Engineer | WSO2 Inc. >> <http://www.wso2.com> | >> (M) 94-77-1663314 >> <http://wso2.com/signature> >> > > > -- > *Hinduja Balasubramaniyam* > Software Engineering Intern | WSO2 > > Email : hind...@wso2.com > Mobile : +94766612192 > > > -- *Fathima Dilhasha* | Senior Software Engineer | WSO2 Inc. <http://www.wso2.com> | (M) 94-77-1663314 <http://wso2.com/signature>
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
