[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667051#comment-16667051
 ] 

ASF subversion and git services commented on KYLIN-2723:


Commit 78b17f5a70fb2e5d66e9cce4def79e5adb377878 in kylin's branch 
refs/heads/master from Zhong
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=78b17f5 ]

KYLIN-2723 fix potential concurrent issue when add rpc statistics


> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Wang Ken
>Priority: Major
> Fix For: v2.3.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[liyang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307334#comment-16307334
 ] 

liyang commented on KYLIN-2723:
---

FYI. I have renamed config "kylin.core.metrics.xxx" to "kylin.metrics.xxx" for 
consistency.

> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Fix For: v2.3.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[Zhong Yanghong (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16236892#comment-16236892
 ] 

Zhong Yanghong commented on KYLIN-2723:
---

The related pull request is https://github.com/apache/kylin/pull/79. And it's 
merged to master branch.

> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[Zhong Yanghong (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121640#comment-16121640
 ] 

Zhong Yanghong commented on KYLIN-2723:
---

This patch combined the patch at [KYLIN-2722 | 
https://issues.apache.org/jira/browse/KYLIN-2722]. It's based on existing 
{{QueryMetricsFacade}} to report query related metrics events to multiple 
media. And It's based on a new class {{JobMetricsFacade}} to report job related 
metrics events to multiple media.

> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2722+APACHE-KYLIN-2723.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[liyang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100374#comment-16100374
 ] 

liyang commented on KYLIN-2723:
---

Think we have settled on dropwizard on KYLIN-2721.

> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2723+APACHE-KYLIN-2722.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[Zhong Yanghong (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095943#comment-16095943
 ] 

Zhong Yanghong commented on KYLIN-2723:
---

Hi guys,

This is for voting which metrics framework kylin metrics should be based on.


Best regards,
Yanghong Zhong

Email: yangzh...@ebay.com
Mobile: +86 13706747741


On 7/21/17, 4:09 PM, "liyang (JIRA)"  wrote:


[ 
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FKYLIN-2723%3Fpage%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%26focusedCommentId%3D16095934%23comment-16095934=02%7C01%7Cyangzhong%40ebay.com%7Ccd5b5627a5244b6d918908d4d00fd11a%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C636362213728751004=88my0zX%2BbGqx96CQbJ1TyQhe5k1WXprUEAuqicC3oTU%3D=0
 ] 

liyang commented on KYLIN-2723:
---

Can we start by listing out the options and pros & cons? What I heard of so 
far are:
- Yammer (Coda Hale)
- Hadoop Metrics2

Are different are they? Or they are simply interchangable?

> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: 
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FKYLIN-2723=02%7C01%7Cyangzhong%40ebay.com%7Ccd5b5627a5244b6d918908d4d00fd11a%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C636362213728751004=H1dfcr%2B%2B2LkMj2yxSXFB26tGnTwz9eSnZrA8PMqqrEE%3D=0
> Project: Kylin
>  Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2723+APACHE-KYLIN-2722.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)




> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2723+APACHE-KYLIN-2722.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[liyang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095934#comment-16095934
 ] 

liyang commented on KYLIN-2723:
---

Can we start by listing out the options and pros & cons? What I heard of so far 
are:
- Yammer (Coda Hale)
- Hadoop Metrics2

Are different are they? Or they are simply interchangable?

> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2723+APACHE-KYLIN-2722.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[liyang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095929#comment-16095929
 ] 

liyang commented on KYLIN-2723:
---

Let's reach consensus on the metrics framework before commit any code. The 
worst thing is we end up with multiple metrics frameworks in the Kylin system.

> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2723+APACHE-KYLIN-2722.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[Zhong Yanghong (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089888#comment-16089888
 ] 

Zhong Yanghong commented on KYLIN-2723:
---

Hi [~hn5092], here's the patch for {{ActiveReservoir}} and reporters. Later 
I'll integrate this metrics framework with query engine & job engine to collect 
related metrics.

> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2723+APACHE-KYLIN-2722.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[Zhong Yanghong (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086905#comment-16086905
 ] 

Zhong Yanghong commented on KYLIN-2723:
---

hi [~hn5092], the metrics we made are based on yammer metrics, different from 
hadoop metrics. We extend yammer metrics by introducing a new measure, called 
ActiveReservoir, which acting as a transfer station to transfer metrics to 
multiple reporters. 
We implemented two reporters. One is to hive. And another is to kafka. Then we 
can build system cube based on the metrics in hive or in kafka. I'm preparing 
patches for these and will finish in near days.

> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KYLIN-2723) Introduce metrics collector for query & job metrics

[yiming.xu (JIRA)]

[ 
https://issues.apache.org/jira/browse/KYLIN-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085482#comment-16085482
 ] 

yiming.xu commented on KYLIN-2723:
--

hi,Zhong Yanghong!I also made a metric, how do you design the metric, is 
similar to hadoop metric, I have done a similar, would like to discuss with you

> Introduce metrics collector for query & job metrics
> ---
>
> Key: KYLIN-2723
> URL: https://issues.apache.org/jira/browse/KYLIN-2723
> Project: Kylin
>  Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)