+1 for the feature. +1 for an abstracted way of metrics library integration.
Some additional thoughts on this: - We might have implications of adding a set of dependencies from drop wizard into the engine core ( guava versions etc). - Flink as pointed out by Thomas seems to have taken an interesting approach to metrics ( apart from supporting multiple reporters ) - Any reporter is only enabled if the relevant jar is put in the classpath. - There seems to be two worlds for generating the metric names. 1. Dot notation separated ( or some separator thereof. Ex: dropwizard style) and 2. high dimensional metric names using key value pair tags. ( prometheus style). We might have to "transform" the dot hierarchical notation to a key value pair based notation based on the reporter that is chosen by the end user. - The scope of the implementation is too big as I understand it and perhaps this feature needs to be done in multiple JIRAs. Also malhar is still at java-7 while engine is at java-8. Some questions: - Is there a plan to expose the metric via a jetty end point on each JVM ? ( if configured ) - How do we plan to handle dynamic partitioning of operators for metrics ? ( i.e possibly short-lived JVMs ) - This feature implies that we will no longer support equivalent of ComplexType of Autometrics ? - This feature also implies that we will no longer support autometric aggregators ? ( and leave this to the metric tools functionality) Regards, Ananth On Thu, May 10, 2018 at 12:37 PM, Chinmay Kolhatkar < chinmaykolhatka...@gmail.com> wrote: > +1 for approach 1. As Thomas mentioned, I think there metrics layer should > be abstracted out from its implementation. This way one can plugin > different metrics systems in apex. > > Also keep in mind that there is a lot of code which uses AutoMetrics > annotations. We should help a smooth transition from that. As next release > will be a major version release, this is a good opportunity for getting rid > of old AutoMetrics and counters functionality. > > Regards, > Chinmay. > > > On Thu, 10 May 2018, 4:47 am Vlad Rozov, <vro...@apache.org> wrote: > > > +1 for the #1 proposal. > > > > Thank you, > > > > Vlad > > > > On 5/9/18 07:14, Thomas Weise wrote: > > > +1 for the initiative > > > > > > Some thoughts: > > > > > > - it is probably good to retain a level of abstraction, to avoid direct > > > dependency on dropwizard > > > - support programmatic metric creation (not just annotations) > > > - remove deprecated counter and auto-metric code and migrate operators > to > > > use new API > > > - which metric reporting systems will be supported out of the box > > > > > > You can also take a look at how this was structured in Flink: > > > > > https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/ > metrics.html > > > > > > Thanks, > > > Thomas > > > > > > > > > On Tue, May 8, 2018 at 8:56 AM, Deepak Narkhede < > mailtodeep...@gmail.com > > > > > > wrote: > > > > > >> Hi Community, > > >> > > >> I want to propose addition of metrics like gauges, meters, counters > and > > >> historgram for the following components. > > >> 1) Addition of metrics for Container Stats. > > >> 2) Addition of metrics for Operator Stats. > > >> 3) Addition of metrics for Stram Application Master stats. > > >> 4) Addition of metrics for JVM related stats for all containers. > > >> > > >> To implement them would be using metrics dropwizard api's. ( > > >> http://metrics.dropwizard.io/) > > >> Use cases: > > >> 1) Can be directly pushed to external visualisation system like > > Graphite. > > >> 2) Can be viewed in visualVM tools through JMX. > > >> 3) Can be outputted to console. > > >> 4) It is also possible to push the metrics to custom sink. > > >> > > >> We will also need to write sinks and reporter, if required for custom > > >> sinks. > > >> > > >> Design/Implementation approach: > > >> Way #1: > > >> 1) Create new annotations like @MetricTypeGauge, @MetricTypeMeter, > > >> @MetricTypeCounter, @MetricTypeHistogram. They can be both fields and > > >> methods. > > >> 2) Add them to respective methods or fields like StreamingContainer, > > >> StreamingAppMasterService for extraction of relevant metrics. > > >> 3) While Node creation ( InputNode/GeneticNode/OiONode), we create > and > > >> initialise the metrics registry depending on components. > > >> 4) While collectMetrics() part of operator runner thread ( > InputNode.run > > >> /GenericNode.run), we actually invoke the annotations methods and > > collect > > >> different types of metrics. > > >> 5) We can have a sink which pushes the metrics to reporter like > Console, > > >> JMX etc. > > >> > > >> Way #2: > > >> Use existing AutoMetrics annotations, convert some metrics to > different > > >> types like gauge, counter etc..But this cannot be done generically as > we > > >> don't know the types. Still more investigation is going on this > > approach. > > >> > > >> I would prefer first way. > > >> > > >> Note: There are some complications, if two operators are deployed on > > same > > >> jvm conatiner. But I think it can be resolved by creating two > different > > >> metrics registry with unique id from JVM. > > >> > > >> Let me know your thoughts on this. > > >> > > >> Thanks, > > >> Deepak > > >> > > > > >
