Hi Chintana, It's possible to turn on/off metrics at runtime via JMX. I have added a JMX Bean to change the Metric levels. So, if the level is set to OFF, the metrics collection will be stopped. This level can be set initially via a system property at startup. As you mentioned, we cannot set system property externally at runtime and therefore I used JMX to avoid any server restarts when changing the levels.
Right now, this metric level is applied to all metrics. However Azeez suggested that we should allow setting levels for each Metrics, so that we can enable/disable a set of metrics (similar to log4j). This part is not yet implemented. Thanks! Best regards, On Tue, Jan 27, 2015 at 9:04 PM, Chintana Wilamuna <chint...@wso2.com> wrote: > Hi Isuru, > > Is it possible to turn on/off metrics using a runtime property? Correct me > if I'm wrong, using a system property for metrics you have to restart the > server for metrics to be turned on/off? > > -Chintana > > On Tue, Jan 27, 2015 at 1:42 AM, Isuru Perera <isu...@wso2.com> wrote: > >> Hi Paul, >> >> Thanks for the suggestions. >> >> I just had a quick chat with Johann. Since there is some more time for >> next IS release, we can start integrating metrics with IS. I'll set up a >> meeting with IS team. >> >> As you mentioned, it would be great if we can integrate Metrics to ESB. >> As I know, ESB Team is in final stages for 4.9.0 release and I think we >> will have to integrate metrics after ESB 4.9.0 release. Kasun, WDYT? >> >> Thanks! >> >> Best Regards, >> >> On Fri, Jan 23, 2015 at 12:34 PM, Paul Fremantle <p...@wso2.com> wrote: >> >>> I suggest one of the following: >>> >>> * The ESB already has very advanced metrics. So a good test would be to >>> see if we can get all of those easily with the new component. That would >>> quickly get the new component up to a high quality. >>> >>> * The Identity Server market is becoming much more performance >>> sensitive, so metrics in IS would be really valuable. >>> >>> Paul >>> >>> On 23 January 2015 at 06:59, Isuru Perera <isu...@wso2.com> wrote: >>> >>>> Hi all, >>>> >>>> This is just a quick update on Metrics integration. >>>> >>>> - We decided to use a separate repository for metrics [1]. The main >>>> reason is that we want to integrate Metrics to upcoming releases and it >>>> will take a longer time for products to use a new Carbon Kernel version. >>>> - Developed a new JDBC Reporter for Metrics [2] >>>> - Now I'm working on a new UI for Metrics visualization. I >>>> experimented with other tools and we decided to have a lightweight UI, >>>> which can be integrated to WSO2 Products. I'm planning to use the data >>>> from >>>> a database and that's the main reason to develop a new JDBC reporter. >>>> I'll >>>> share the details of the UI later. >>>> >>>> Since the core part for Metrics API is ready, we can start integrating >>>> to the products. >>>> >>>> Srinath, please let me know which product to start with. Then we can >>>> discuss with relevant product team. >>>> >>>> Thanks! >>>> >>>> Best Regards, >>>> >>>> [1] https://github.com/wso2/carbon-metrics >>>> >>>> [2] >>>> https://github.com/wso2/carbon-metrics/blob/master/components/org.wso2.carbon.metrics.jdbc.reporter/src/main/java/org/wso2/carbon/metrics/reporter/JDBCReporter.java >>>> >>>> On Tue, Dec 23, 2014 at 3:35 PM, Isuru Perera <isu...@wso2.com> wrote: >>>> >>>>> Hi Sameera, >>>>> >>>>> I sent a PR to carbon4-kernel repository [1]. Could you please review >>>>> and merge it? I created CARBON-15115 [2] for that. >>>>> >>>>> Thanks! >>>>> >>>>> [1] https://github.com/wso2/carbon4-kernel/pull/125 >>>>> [2] https://wso2.org/jira/browse/CARBON-15115 >>>>> >>>>> On Wed, Dec 3, 2014 at 1:53 PM, Ramith Jayasinghe <ram...@wso2.com> >>>>> wrote: >>>>> >>>>>> Yep. This needs to be evaluated on loaded environments. >>>>>> >>>>>> On Wed, Dec 3, 2014 at 1:39 PM, Isuru Perera <isu...@wso2.com> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> I have committed what I did so far to the WSO2 SVN scratch area [1]. >>>>>>> >>>>>>> There are two components. >>>>>>> >>>>>>> 1. org.wso2.carbon.metrics.manager - Contains the WSO2 Metrics >>>>>>> API, which will be used in the WSO2 platform. >>>>>>> 2. org.wso2.carbon.metrics.impl - The implementation of Metric >>>>>>> Service, which will internally use the Metrics Java Library [2]. >>>>>>> >>>>>>> The main reason for having our own API is to introduce the support >>>>>>> for different metrics levels. This will also make sure that we are not >>>>>>> tightly coupled with Metrics Java library. >>>>>>> >>>>>>> The org.wso2.carbon.metrics.manager.MetricManager [3] class has the >>>>>>> main API. This class is similar to the Metrics' MetricRegistry class [4] >>>>>>> and I have taken the helper methods to generate names etc. >>>>>>> >>>>>>> Yesterday, I organized a code review and following are the notes: >>>>>>> >>>>>>> Participants: Azeez, Sameera, Ramith. >>>>>>> >>>>>>> Code review was based on revision 210170 [5] >>>>>>> >>>>>>> - Azeez mentioned that having a global configuration for Level >>>>>>> [6] might not be useful. Since we allow to create different Metrics >>>>>>> with >>>>>>> different levels, we need to have a way to specify level for each >>>>>>> Metric >>>>>>> similar to Log4J. Configurations should be decided initially rather >>>>>>> than >>>>>>> introducing later. >>>>>>> - Ramith asked whether we can support annotations. Metrics [2] >>>>>>> support annotations for Spring with Spring's AOP support. Besides, >>>>>>> the >>>>>>> annotation will help to measure up to method level only. It was >>>>>>> decided >>>>>>> that we can use the APIs directly as we can use the given APIs >>>>>>> whenever we >>>>>>> need to measure. >>>>>>> - The use of ConcurrentHashMap need to be reevaluated. The >>>>>>> MetricManager also uses a ConcurrentHashMap similar to >>>>>>> MetricRegistry to >>>>>>> avoid duplicate Metric objects. We must test the Metrics API in high >>>>>>> concurrent scenarios. >>>>>>> - We decided to integrate the metrics components to Carbon >>>>>>> Kernel first. I will start integrating the metrics to Carbon Kernel >>>>>>> 4.4 in >>>>>>> a different branch. >>>>>>> - When logging, isErrorEnabled may not be necessary [7], but >>>>>>> it's okay to have it. >>>>>>> >>>>>>> Azeez, Sameera, Ramith, please add if I missed anything. >>>>>>> >>>>>>> In addition to the code review, I had a chat with Srinath yesterday >>>>>>> morning. >>>>>>> >>>>>>> We discussed that we must have a UI for Metrics. I will share the >>>>>>> details on that later. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> Best Regards, >>>>>>> >>>>>>> [1] https://svn.wso2.org/repos/wso2/scratch/metrics >>>>>>> [2] https://dropwizard.github.io/metrics >>>>>>> [3] >>>>>>> https://svn.wso2.org/repos/wso2/scratch/metrics/components/org.wso2.carbon.metrics.manager/src/main/java/org/wso2/carbon/metrics/manager/MetricManager.java >>>>>>> [4] >>>>>>> https://github.com/dropwizard/metrics/blob/master/metrics-core/src/main/java/com/codahale/metrics/MetricRegistry.java >>>>>>> [5] http://wso2.org/svn/browse/wso2?view=revision&revision=210170 >>>>>>> [6] >>>>>>> http://wso2.org/svn/browse/wso2/scratch/metrics/components/org.wso2.carbon.metrics.impl/src/test/resources/metrics.xml?view=markup&pathrev=210170 >>>>>>> [7] >>>>>>> http://wso2.org/svn/browse/wso2/scratch/metrics/components/org.wso2.carbon.metrics.manager/src/main/java/org/wso2/carbon/metrics/manager/internal/MetricManagerComponent.java?view=markup&pathrev=210170 >>>>>>> >>>>>>> On Tue, Nov 18, 2014 at 12:11 PM, Isuru Perera <isu...@wso2.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Me & Srinath discussed a bit more on implementation. Following are >>>>>>>> the notes. >>>>>>>> >>>>>>>> - We need to have different levels for each metric similar to >>>>>>>> Log4J levels. The API for metric will have a way to specify the >>>>>>>> level. >>>>>>>> - There should be a way to view historical data. By default, we >>>>>>>> can use a log file. >>>>>>>> - There should be a configuration file for metrics. The enable >>>>>>>> option will be part of the configuration file. But it can be >>>>>>>> overridden by >>>>>>>> a System property. >>>>>>>> >>>>>>>> Srinath, >>>>>>>> >>>>>>>> Please add if I missed anything. Also regarding the reporting >>>>>>>> options, the metrics provides support for Ganglia & Graphite [1]. The >>>>>>>> in-built support for HTTP reporting might not be directly useful for us >>>>>>>> [2]. I will check on these and see how we can get the data. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> Best Regards, >>>>>>>> [1] >>>>>>>> https://dropwizard.github.io/metrics/3.1.0/getting-started/#other-reporting >>>>>>>> [2] >>>>>>>> https://dropwizard.github.io/metrics/3.1.0/manual/servlets/#manual-servlets >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Nov 18, 2014 at 7:53 AM, Srinath Perera <srin...@wso2.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Look good Isuru!! >>>>>>>>> >>>>>>>>> On Thu, Nov 13, 2014 at 6:58 PM, Isuru Perera <isu...@wso2.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> I played with following performance libraries. >>>>>>>>>> >>>>>>>>>> - Metrics >>>>>>>>>> - Parfait >>>>>>>>>> - JAMon >>>>>>>>>> - Java Simon >>>>>>>>>> - Perf4J >>>>>>>>>> >>>>>>>>>> I did a simple comparison and blogged [1]. >>>>>>>>>> >>>>>>>>>> As Srinath mentioned, the Metrics library is more suitable for >>>>>>>>>> this proposal. >>>>>>>>>> >>>>>>>>>> It supports various measuring instruments like Meters, Counters, >>>>>>>>>> Histograms, Timers etc. >>>>>>>>>> >>>>>>>>>> Metrics supports JMX reporting and we should be able to write our >>>>>>>>>> own reporting implementations easily. We can use a reporting >>>>>>>>>> implementation >>>>>>>>>> to integrate with BAM. >>>>>>>>>> >>>>>>>>>> There is no support to enable/disable metrics. Since this is an >>>>>>>>>> important feature for us, I'm thinking of introducing a custom >>>>>>>>>> component >>>>>>>>>> for performance monitoring. >>>>>>>>>> >>>>>>>>>> The custom component will support to enable/disable performance >>>>>>>>>> monitoring via system property and JMX. The WSO2 components will use >>>>>>>>>> the >>>>>>>>>> new performance monitoring component and we will be using the >>>>>>>>>> Metrics only >>>>>>>>>> inside the new component. >>>>>>>>>> >>>>>>>>>> I'm working on this and for testing we need to select a WSO2 >>>>>>>>>> product. I'm thinking of adding the performance probes to WSO2 API >>>>>>>>>> Manager >>>>>>>>>> first. Since WSO2 CEP is also mentioned, we can have a look at that >>>>>>>>>> also. >>>>>>>>>> >>>>>>>>>> Please share any suggestions or feedback. >>>>>>>>>> >>>>>>>>>> I will update the thread once I create the initial design. >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> >>>>>>>>>> Best Regards, >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> http://isuru-perera.blogspot.com/2014/11/java-performance-monitoring-libraries.html >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Sep 29, 2014 at 10:00 AM, Srinath Perera < >>>>>>>>>> srin...@wso2.com> wrote: >>>>>>>>>> >>>>>>>>>>> Thanks Thomas!! >>>>>>>>>>> >>>>>>>>>>> Paul, I looked at all 3 and really liked what Thomas sent ( >>>>>>>>>>> https://dropwizard.github.io/metrics/3.1.0/getting-started/). >>>>>>>>>>> Licence is Apache 2. >>>>>>>>>>> >>>>>>>>>>> I think I will try this out with a training project. >>>>>>>>>>> >>>>>>>>>>> --Srinath >>>>>>>>>>> >>>>>>>>>>> On Sat, Sep 27, 2014 at 11:37 AM, Thomas Wieger < >>>>>>>>>>> developer.wie...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> please have a look on the metrics lib, which quite nicely >>>>>>>>>>>> provides support for capturing different kinds of statistics. see >>>>>>>>>>>> https://dropwizard.github.io/metrics/3.1.0/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> regards, >>>>>>>>>>>> >>>>>>>>>>>> thomas >>>>>>>>>>>> >>>>>>>>>>>> Am Freitag, 26. September 2014 schrieb Sajith Ravindra : >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> IMHO, CEP is another product where this kind of probes can be >>>>>>>>>>>>> really useful as we have to deal with performance related >>>>>>>>>>>>> requirements. As >>>>>>>>>>>>> there's a major refactoring going on in siddhi I think we can we >>>>>>>>>>>>> plant >>>>>>>>>>>>> probes like this into the Siddhi engine. It will be useful to >>>>>>>>>>>>> evaluate the >>>>>>>>>>>>> re-factored Sindhi engines performance as well as to figure out >>>>>>>>>>>>> performance >>>>>>>>>>>>> bottlenecks in the future. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> *,Sajith Ravindra* >>>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>>> WSO2 Inc.; http://wso2.com >>>>>>>>>>>>> lean.enterprise.middleware >>>>>>>>>>>>> >>>>>>>>>>>>> mobile: +94 77 2273550 >>>>>>>>>>>>> blog: http://sajithr.blogspot.com/ >>>>>>>>>>>>> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Sep 26, 2014 at 8:38 AM, Srinath Perera < >>>>>>>>>>>>> srin...@wso2.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> We have some pretty nice probes already in ESB, that let you >>>>>>>>>>>>>> look into what is going in the same running in production. I >>>>>>>>>>>>>> think we need >>>>>>>>>>>>>> to do this generally and build it to other products. That should >>>>>>>>>>>>>> reduce >>>>>>>>>>>>>> time we spent debugging issues significantly. >>>>>>>>>>>>>> >>>>>>>>>>>>>> My proposal is to build a probe Lib that looks like >>>>>>>>>>>>>> following. >>>>>>>>>>>>>> >>>>>>>>>>>>>> There are two kinds of things you need to collect. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. Throughput at give point of code (how fast calls goes >>>>>>>>>>>>>> though) >>>>>>>>>>>>>> 2. Latency between two points. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> We will have two types of Probes. Code would look like >>>>>>>>>>>>>> following. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Probe probe = new Probe("Name", "throughput", timeDuration); >>>>>>>>>>>>>> .. >>>>>>>>>>>>>> probe.recordThroughput(); >>>>>>>>>>>>>> >>>>>>>>>>>>>> Probe probe = new Probe("Name", "latency", timeDuration, >>>>>>>>>>>>>> logLevel); >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> long id = probe.startTicking() // this so same probe can be >>>>>>>>>>>>>> used by many threads >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> probe.endTicking(id); >>>>>>>>>>>>>> >>>>>>>>>>>>>> Probe will summarise data over given duration and expose. We >>>>>>>>>>>>>> need >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. JMX bean >>>>>>>>>>>>>> 2. BAM Agent >>>>>>>>>>>>>> 3. Can turn on, off via JMX agent or via System property >>>>>>>>>>>>>> 4. Have Tool Box >>>>>>>>>>>>>> 5. Have in product UI >>>>>>>>>>>>>> 6. Can configured to write data to logs >>>>>>>>>>>>>> >>>>>>>>>>>>>> Each probe should be very small and should be able to create >>>>>>>>>>>>>> thousands without much effect. (e.g. create one for each >>>>>>>>>>>>>> mediator type) >>>>>>>>>>>>>> >>>>>>>>>>>>>> WDYT? >>>>>>>>>>>>>> >>>>>>>>>>>>>> --Srinath >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> ============================ >>>>>>>>>>>>>> Blog: http://srinathsview.blogspot.com >>>>>>>>>>>>>> twitter:@srinath_perera >>>>>>>>>>>>>> Site: http://people.apache.org/~hemapani/ >>>>>>>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>>>>>>> Phone: 0772360902 >>>>>>>>>>>>>> >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>>>> Architecture@wso2.org >>>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> ============================ >>>>>>>>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>>>>>>>> Site: http://people.apache.org/~hemapani/ >>>>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>>>> Phone: 0772360902 >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Architecture mailing list >>>>>>>>>>> Architecture@wso2.org >>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Isuru Perera >>>>>>>>>> Senior Software Engineer | WSO2, Inc. | http://wso2.com/ >>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>> >>>>>>>>>> about.me/chrishantha >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> ============================ >>>>>>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>>>>>> Site: http://people.apache.org/~hemapani/ >>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>> Phone: 0772360902 >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Isuru Perera >>>>>>>> Senior Software Engineer | WSO2, Inc. | http://wso2.com/ >>>>>>>> Lean . Enterprise . Middleware >>>>>>>> >>>>>>>> about.me/chrishantha >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Isuru Perera >>>>>>> Senior Software Engineer | WSO2, Inc. | http://wso2.com/ >>>>>>> Lean . Enterprise . Middleware >>>>>>> >>>>>>> about.me/chrishantha >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Ramith Jayasinghe >>>>>> Technical Lead >>>>>> WSO2 Inc., http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> >>>>>> E: ram...@wso2.com >>>>>> P: +94 777542851 >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Isuru Perera >>>>> Senior Software Engineer | WSO2, Inc. | http://wso2.com/ >>>>> Lean . Enterprise . Middleware >>>>> >>>>> about.me/chrishantha >>>>> >>>> >>>> >>>> >>>> -- >>>> Isuru Perera >>>> Senior Software Engineer | WSO2, Inc. | http://wso2.com/ >>>> Lean . Enterprise . Middleware >>>> >>>> about.me/chrishantha >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Paul Fremantle >>> CTO and Co-Founder, WSO2 >>> OASIS WS-RX TC Co-chair, Apache Member >>> >>> UK: +44 207 096 0336 >>> >>> blog: http://pzf.fremantle.org >>> twitter.com/pzfreo >>> p...@wso2.com >>> >>> wso2.com Lean Enterprise Middleware >>> >>> Disclaimer: This communication may contain privileged or other >>> confidential information and is intended exclusively for the addressee/s. >>> If you are not the intended recipient/s, or believe that you may have >>> received this communication in error, please reply to the sender indicating >>> that fact and delete the copy you received and in addition, you should not >>> print, copy, retransmit, disseminate, or otherwise use the information >>> contained in this communication. Internet communications cannot be >>> guaranteed to be timely, secure, error or virus-free. The sender does not >>> accept liability for any errors or omissions. >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Isuru Perera >> Senior Software Engineer | WSO2, Inc. | http://wso2.com/ >> Lean . Enterprise . Middleware >> >> about.me/chrishantha >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Chintana Wilamuna > Solutions Architect > WSO2, Inc.; http://wso2.com > lean.enterprise.middleware > > phone: +1 408 429 3321 > blog: http://engwar.com/ > linkedin: http://www.linkedin.com/in/chintanawilamuna > twitter: twitter.com/std_err > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Isuru Perera Senior Software Engineer | WSO2, Inc. | http://wso2.com/ Lean . Enterprise . Middleware about.me/chrishantha
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
