Hi Nandika, We can add a Mbean that exposes similar functionality as explained by Nirmal. Would that help to achieve your requirement?
Regards, Gihan On Fri, Jun 10, 2016 at 10:08 AM, Nirmal Fernando <nir...@wso2.com> wrote: > Hi All, > > I actually used Carbon metrics and added few Mbeans to get the remaining > queue size etc. from the data publisher agent level and DAS level as part > of per tests we're doing for API-M Analytics. > > I think we could build upon these too. Please see the following diffs. > > Agent: > https://github.com/nirmal070125/carbon-analytics-common/commit/1375677fd2a39303e8cbaaea33c6a6a0cdd672dc > > DAS: > https://github.com/nirmal070125/carbon-analytics-common/commit/0d26ba4a6ac732b161584905ff3ecf7ae20fa53e > > On Fri, Jun 10, 2016 at 9:49 AM, Nandika Jayawardana <nand...@wso2.com> > wrote: > >> Hi Gihan, >> >> Do you guys have any plans to expose some MBeans for the DAS publisher. >> Consider a scenario where a user implements a stand-alone DAS publisher to >> publish a large data set ( Eg : archived log files ). In such a scenario, >> we need a way to monitor the progress and to know whether the client >> finished publishing the dataset to DAS. So having some MBeans would be >> really useful for publisher as well. >> >> Regards >> Nandika >> >> On Thu, Jun 9, 2016 at 7:10 PM, Isuru Perera <isu...@wso2.com> wrote: >> >>> Hi, >>> >>> On Thu, Jun 9, 2016 at 5:30 PM, Gihan Anuruddha <gi...@wso2.com> wrote: >>> >>>> Initially we thought about that. But then we decided to go with >>>> normal Mbean expose due following reasons. >>>> 1. To minimize depend on external repositories. >>>> >>> carbon-metrics is recommended for exposing these kind of values. DAS >>> already has carbon-metrics, so I don't think this is an issue. >>> >>>> 2. To list the analytics subdomain under org.wso2.carbon domain. >>>> >>> You can use a meaningful names metrics. You can use a common prefix >>> analytics metrics. >>> >>>> 3. I couldn't find a way to expose operations. >>>> >>> Instead of exposing operations, shall we try to use Gauges with some >>> formatted name with your parameters. For example, you can use following >>> names for getRemainingBufferCapacityInBytes(int tenantId) and >>> int getCurrentQueueSize(int tenantId). >>> >>> org.wso2.carbon.analytics.<tenant-id>.disruptor.remaining-buffer-size >>> org.wso2.carbon.analytics.<tenant-id>.disruptor.current-queue-size >>> >>> This will add more metrics, as there are gauges for every tenant. But I >>> think that should be fine. If you use Metrics, you can also get historical >>> data, which is very useful if you want to debug an issue happened in the >>> past (may be few mins ago). >>> >>> Even the metrics library is available these kinds of requirements. >>> Therefore I think we must use it and try to avoid implementing separate >>> MBeans for any requirements that can be done with Metrics. >>> >>> Thanks! >>> >>> >>> >>>> Regards, >>>> Gihan >>>> >>>> On Thu, Jun 9, 2016 at 5:09 PM, Isuru Perera <isu...@wso2.com> wrote: >>>> >>>>> Any reason for not using Carbon Metrics? >>>>> >>>>> On Thu, Jun 9, 2016 at 5:00 PM, Gihan Anuruddha <gi...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> We have added a couple of MBeans for DAS to expose some debug level >>>>>> information. These Mbeans will list under org.wso2.carbon.analytics >>>>>> subdomain. >>>>>> >>>>>> EVENT_COUNTER [int getCurrentCount()]- This will count all the >>>>>> events that received to DAS regardless of steam or tenant. We want to add >>>>>> this per tenant per stream. But in order to do that we need to add a >>>>>> Map to event persistence path and that might add extra delay to the event >>>>>> saving critical path. Because of that we thought of adding only a >>>>>> counter (AtomicLong). >>>>>> >>>>>> RECEIVER_REMAINING_QUEUE_BUFFER_SIZE_IN_BYTES >>>>>> [ >>>>>> long getRemainingBufferCapacityInBytes(int tenantId), >>>>>> int getCurrentQueueSize(int tenantId)] >>>>>> - You can use this Mbean to get remaining buffer size in disruptor and >>>>>> current queue size. >>>>>> >>>>>> LAST_PROCESSED_TIMESTAMP[long getLastProcessedTimestamp(int tenantId, >>>>>> String id, boolean primary)] >>>>>> - This will return last saved incremental processed timestamp for >>>>>> given spark table. >>>>>> >>>>>> ANALYTICS_SCRIPT_LAST_EXECUTION_START_TIME[long >>>>>> getScriptLastExecutionStartTime(int tenantId, String scriptName)] >>>>>> - This will return latest execution start time of the given Spark >>>>>> script. >>>>>> >>>>>> Please let me know any other information that you think better >>>>>> expose through Mbeans. >>>>>> >>>>>> Regards, >>>>>> Gihan >>>>>> >>>>>> -- >>>>>> W.G. Gihan Anuruddha >>>>>> Senior Software Engineer | WSO2, Inc. >>>>>> M: +94772272595 >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> Architecture@wso2.org >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Isuru Perera >>>>> Associate Technical Lead | WSO2, Inc. | http://wso2.com/ >>>>> Lean . Enterprise . Middleware >>>>> >>>>> about.me/chrishantha >>>>> Contact: +IsuruPereraWSO2 >>>>> <https://www.google.com/+IsuruPereraWSO2/about> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> Architecture@wso2.org >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> W.G. Gihan Anuruddha >>>> Senior Software Engineer | WSO2, Inc. >>>> M: +94772272595 >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Isuru Perera >>> Associate Technical Lead | WSO2, Inc. | http://wso2.com/ >>> Lean . Enterprise . Middleware >>> >>> about.me/chrishantha >>> Contact: +IsuruPereraWSO2 >>> <https://www.google.com/+IsuruPereraWSO2/about> >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Nandika Jayawardana >> WSO2 Inc ; http://wso2.com >> lean.enterprise.middleware >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > > Thanks & regards, > Nirmal > > Team Lead - WSO2 Machine Learner > Associate Technical Lead - Data Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > > -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
