[jira] [Assigned] (EAGLE-1095) restart eagle server
[ https://issues.apache.org/jira/browse/EAGLE-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen reassigned EAGLE-1095: Assignee: Zhao, Qingwen > restart eagle server > > > Key: EAGLE-1095 > URL: https://issues.apache.org/jira/browse/EAGLE-1095 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Dylan Zhao >Assignee: Zhao, Qingwen >Priority: Major > Fix For: v0.5.0 > > > After restarting the eagle server, the newly created site and data will be > reset. How to solve this problem, the work before the service restart is in > vain? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (EAGLE-1095) restart eagle server
[ https://issues.apache.org/jira/browse/EAGLE-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835280#comment-16835280 ] Zhao, Qingwen commented on EAGLE-1095: -- Hi [~DylanZhao], Do you use the in-memory storage in your configuration? I think you can use a persistent one, such as MySQL. > restart eagle server > > > Key: EAGLE-1095 > URL: https://issues.apache.org/jira/browse/EAGLE-1095 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Dylan Zhao >Priority: Major > Fix For: v0.5.0 > > > After restarting the eagle server, the newly created site and data will be > reset. How to solve this problem, the work before the service restart is in > vain? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (EAGLE-1096) getJSONArray may cause exception, optJSONArray should be used
[
https://issues.apache.org/jira/browse/EAGLE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on EAGLE-1096 started by Zhao, Qingwen.
> getJSONArray may cause exception, optJSONArray should be used
> -
>
> Key: EAGLE-1096
> URL: https://issues.apache.org/jira/browse/EAGLE-1096
> Project: Eagle
> Issue Type: Bug
>Reporter: bd2019us
>Assignee: Zhao, Qingwen
>Priority: Major
> Attachments: 1.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Hello,
> The method call jsonBeansObject.getJSONArray("beans") may throw
> JSONException when the key does not exist, which will crash the program. To
> ensure safety, optJSONArray is recommended to use, which will avoid the
> exception in case no such key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (EAGLE-1096) getJSONArray may cause exception, optJSONArray should be used
[
https://issues.apache.org/jira/browse/EAGLE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835276#comment-16835276
]
Zhao, Qingwen commented on EAGLE-1096:
--
Hi [~bd2019us],
Thanks for contributing the fixing. It will be appreciated if you could
continue to refine your change.
> getJSONArray may cause exception, optJSONArray should be used
> -
>
> Key: EAGLE-1096
> URL: https://issues.apache.org/jira/browse/EAGLE-1096
> Project: Eagle
> Issue Type: Bug
>Reporter: bd2019us
>Priority: Major
> Attachments: 1.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Hello,
> The method call jsonBeansObject.getJSONArray("beans") may throw
> JSONException when the key does not exist, which will crash the program. To
> ensure safety, optJSONArray is recommended to use, which will avoid the
> exception in case no such key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (EAGLE-1096) getJSONArray may cause exception, optJSONArray should be used
[
https://issues.apache.org/jira/browse/EAGLE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen reassigned EAGLE-1096:
Assignee: (was: Zhao, Qingwen)
> getJSONArray may cause exception, optJSONArray should be used
> -
>
> Key: EAGLE-1096
> URL: https://issues.apache.org/jira/browse/EAGLE-1096
> Project: Eagle
> Issue Type: Bug
>Reporter: bd2019us
>Priority: Major
> Attachments: 1.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Hello,
> The method call jsonBeansObject.getJSONArray("beans") may throw
> JSONException when the key does not exist, which will crash the program. To
> ensure safety, optJSONArray is recommended to use, which will avoid the
> exception in case no such key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (EAGLE-1096) getJSONArray may cause exception, optJSONArray should be used
[
https://issues.apache.org/jira/browse/EAGLE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen reassigned EAGLE-1096:
Assignee: Zhao, Qingwen
> getJSONArray may cause exception, optJSONArray should be used
> -
>
> Key: EAGLE-1096
> URL: https://issues.apache.org/jira/browse/EAGLE-1096
> Project: Eagle
> Issue Type: Bug
>Reporter: bd2019us
>Assignee: Zhao, Qingwen
>Priority: Major
> Attachments: 1.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Hello,
> The method call jsonBeansObject.getJSONArray("beans") may throw
> JSONException when the key does not exist, which will crash the program. To
> ensure safety, optJSONArray is recommended to use, which will avoid the
> exception in case no such key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (EAGLE-1097) CorrelationSpout存在性能问题
[ https://issues.apache.org/jira/browse/EAGLE-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen reassigned EAGLE-1097: Assignee: Zhao, Qingwen > CorrelationSpout存在性能问题 > -- > > Key: EAGLE-1097 > URL: https://issues.apache.org/jira/browse/EAGLE-1097 > Project: Eagle > Issue Type: Improvement > Components: Core::Eagle Server >Affects Versions: v0.3.0 >Reporter: zhangchi >Assignee: Zhao, Qingwen >Priority: Major > Fix For: v0.3.1 > > Attachments: Catch(04-23-09-26-01).jpg, Catch633F(04-23-09-26-01).jpg > > > I ran into a problem where performance dropped dramatically when I switched > on two strategies.These two policies subscribe to two kafka topics, > respectively. I looked at processes that were using too much CPU, and found > that the performance bottlenecks were in the spout send phase.The topics I > subscribed to were divided into 10 sections, and the numOfSpoutTasks I > configured was 20. How shall I adjust the performance problem? > --- > 我遇到了一个问题，在我开启了两个策略的时候，性能会急剧下降。这两个策略分别订阅两个kafka > topic,我查看了CPU使用过高的进程，发现性能瓶颈都是在spout发送这一环节。我订阅的topic分别有10个分区，我配置的numOfSpoutTasks是20.请问我要怎么调整这个性能问题。 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (EAGLE-1097) CorrelationSpout存在性能问题
[ https://issues.apache.org/jira/browse/EAGLE-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on EAGLE-1097 started by Zhao, Qingwen. > CorrelationSpout存在性能问题 > -- > > Key: EAGLE-1097 > URL: https://issues.apache.org/jira/browse/EAGLE-1097 > Project: Eagle > Issue Type: Improvement > Components: Core::Eagle Server >Affects Versions: v0.3.0 >Reporter: zhangchi >Assignee: Zhao, Qingwen >Priority: Major > Fix For: v0.3.1 > > Attachments: Catch(04-23-09-26-01).jpg, Catch633F(04-23-09-26-01).jpg > > > I ran into a problem where performance dropped dramatically when I switched > on two strategies.These two policies subscribe to two kafka topics, > respectively. I looked at processes that were using too much CPU, and found > that the performance bottlenecks were in the spout send phase.The topics I > subscribed to were divided into 10 sections, and the numOfSpoutTasks I > configured was 20. How shall I adjust the performance problem? > --- > 我遇到了一个问题，在我开启了两个策略的时候，性能会急剧下降。这两个策略分别订阅两个kafka > topic,我查看了CPU使用过高的进程，发现性能瓶颈都是在spout发送这一环节。我订阅的topic分别有10个分区，我配置的numOfSpoutTasks是20.请问我要怎么调整这个性能问题。 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (EAGLE-1097) CorrelationSpout存在性能问题
[ https://issues.apache.org/jira/browse/EAGLE-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835273#comment-16835273 ] Zhao, Qingwen commented on EAGLE-1097: -- Hi [~zhangchi], You can either increase the topic partitions or reduce the numberOfSpoutTasks. If you the partition number < the totalTasks, some tasks will be idle. You can check the source code at here [https://github.com/apache/storm/blob/0.9.x-branch/external/storm-kafka/src/jvm/storm/kafka/ZkCoordinator.java#L80] > CorrelationSpout存在性能问题 > -- > > Key: EAGLE-1097 > URL: https://issues.apache.org/jira/browse/EAGLE-1097 > Project: Eagle > Issue Type: Improvement > Components: Core::Eagle Server >Affects Versions: v0.3.0 >Reporter: zhangchi >Priority: Major > Fix For: v0.3.1 > > Attachments: Catch(04-23-09-26-01).jpg, Catch633F(04-23-09-26-01).jpg > > > I ran into a problem where performance dropped dramatically when I switched > on two strategies.These two policies subscribe to two kafka topics, > respectively. I looked at processes that were using too much CPU, and found > that the performance bottlenecks were in the spout send phase.The topics I > subscribed to were divided into 10 sections, and the numOfSpoutTasks I > configured was 20. How shall I adjust the performance problem? > --- > 我遇到了一个问题，在我开启了两个策略的时候，性能会急剧下降。这两个策略分别订阅两个kafka > topic,我查看了CPU使用过高的进程，发现性能瓶颈都是在spout发送这一环节。我订阅的topic分别有10个分区，我配置的numOfSpoutTasks是20.请问我要怎么调整这个性能问题。 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (EAGLE-1051) Check if a publisher can be deleted
[ https://issues.apache.org/jira/browse/EAGLE-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1051: - Affects Version/s: (was: v0.5.1) v0.5.0 > Check if a publisher can be deleted > > > Key: EAGLE-1051 > URL: https://issues.apache.org/jira/browse/EAGLE-1051 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.1 > > > A publisher can be deleted only when there are no policies using it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1051) Check if a publisher can be deleted
[ https://issues.apache.org/jira/browse/EAGLE-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1051: - Fix Version/s: (was: v0.5.0) v0.5.1 > Check if a publisher can be deleted > > > Key: EAGLE-1051 > URL: https://issues.apache.org/jira/browse/EAGLE-1051 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.1 > > > A publisher can be deleted only when there are no policies using it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1051) Check if a publisher can be deleted
[ https://issues.apache.org/jira/browse/EAGLE-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1051: - Affects Version/s: (was: v0.5.0) v0.5.1 > Check if a publisher can be deleted > > > Key: EAGLE-1051 > URL: https://issues.apache.org/jira/browse/EAGLE-1051 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.1 > > > A publisher can be deleted only when there are no policies using it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (EAGLE-1029) Wrong groupSpec is generated when more than one alert engine topology
[ https://issues.apache.org/jira/browse/EAGLE-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen closed EAGLE-1029. Resolution: Fixed > Wrong groupSpec is generated when more than one alert engine topology > -- > > Key: EAGLE-1029 > URL: https://issues.apache.org/jira/browse/EAGLE-1029 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Critical > Fix For: v0.5.0 > > > According to the schedule info returned by /rest/metadata/schedulestates > In alertSpecs, new policies are assigned to alertBolt 9, 18, 19, 1, 12 of the > second alert engine topology. While in groupSpecs, the worker queue(alert > bolts) allocated for each policy is not [ 9, 18, 19, 1, 12]. Actually, it > adds all working queues, including those of other alert engine topology -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-809) Hide Kafka sink configuration used by alert engine
[ https://issues.apache.org/jira/browse/EAGLE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-809: Affects Version/s: (was: v0.5.1) v0.5.0 > Hide Kafka sink configuration used by alert engine > -- > > Key: EAGLE-809 > URL: https://issues.apache.org/jira/browse/EAGLE-809 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.1 > > > * Hide Kafka topic creation used by alert engine > * Hide Kafka broker list used by alert engine -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-809) Hide Kafka sink configuration used by alert engine
[ https://issues.apache.org/jira/browse/EAGLE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-809: Affects Version/s: (was: v0.5.0) v0.5.1 > Hide Kafka sink configuration used by alert engine > -- > > Key: EAGLE-809 > URL: https://issues.apache.org/jira/browse/EAGLE-809 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.1 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.1 > > > * Hide Kafka topic creation used by alert engine > * Hide Kafka broker list used by alert engine -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1029) Wrong groupSpec is generated when more than one alert engine topology
[ https://issues.apache.org/jira/browse/EAGLE-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1029: - Affects Version/s: (was: v0.5.1) v0.5.0 > Wrong groupSpec is generated when more than one alert engine topology > -- > > Key: EAGLE-1029 > URL: https://issues.apache.org/jira/browse/EAGLE-1029 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Critical > Fix For: v0.5.0 > > > According to the schedule info returned by /rest/metadata/schedulestates > In alertSpecs, new policies are assigned to alertBolt 9, 18, 19, 1, 12 of the > second alert engine topology. While in groupSpecs, the worker queue(alert > bolts) allocated for each policy is not [ 9, 18, 19, 1, 12]. Actually, it > adds all working queues, including those of other alert engine topology -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1029) Wrong groupSpec is generated when more than one alert engine topology
[ https://issues.apache.org/jira/browse/EAGLE-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1029: - Fix Version/s: v0.5.0 > Wrong groupSpec is generated when more than one alert engine topology > -- > > Key: EAGLE-1029 > URL: https://issues.apache.org/jira/browse/EAGLE-1029 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Critical > Fix For: v0.5.0 > > > According to the schedule info returned by /rest/metadata/schedulestates > In alertSpecs, new policies are assigned to alertBolt 9, 18, 19, 1, 12 of the > second alert engine topology. While in groupSpecs, the worker queue(alert > bolts) allocated for each policy is not [ 9, 18, 19, 1, 12]. Actually, it > adds all working queues, including those of other alert engine topology -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1029) Wrong groupSpec is generated when more than one alert engine topology
[ https://issues.apache.org/jira/browse/EAGLE-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1029: - Affects Version/s: (was: v0.5.0) v0.5.1 > Wrong groupSpec is generated when more than one alert engine topology > -- > > Key: EAGLE-1029 > URL: https://issues.apache.org/jira/browse/EAGLE-1029 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.1 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Critical > > According to the schedule info returned by /rest/metadata/schedulestates > In alertSpecs, new policies are assigned to alertBolt 9, 18, 19, 1, 12 of the > second alert engine topology. While in groupSpecs, the worker queue(alert > bolts) allocated for each policy is not [ 9, 18, 19, 1, 12]. Actually, it > adds all working queues, including those of other alert engine topology -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1031) add intervalMin setting in the metric preview page
[
https://issues.apache.org/jira/browse/EAGLE-1031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1031:
-
Affects Version/s: (was: v0.5.0)
v0.5.1
> add intervalMin setting in the metric preview page
> ---
>
> Key: EAGLE-1031
> URL: https://issues.apache.org/jira/browse/EAGLE-1031
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.5.1
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
>
> By default, the intervalMin is calculated by
> {code}
> intervalMin = 5min, |queryEndTime - queryStartTime| <= 6h
> 15min, |queryEndTime - queryStartTime| <= 24h
> 30min, |queryEndTime - queryStartTime| <= 7d
> 60min, |queryEndTime - queryStartTime| <= 14d
> 1day, |queryEndTime - queryStartTime| > 14d
> {code}
> In some cases, users want to view metric in minute level. Eagle should
> support to customize this parameter.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (EAGLE-1044) Support policy import using a policy prototype
[ https://issues.apache.org/jira/browse/EAGLE-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1044: - Affects Version/s: v0.5.1 > Support policy import using a policy prototype > -- > > Key: EAGLE-1044 > URL: https://issues.apache.org/jira/browse/EAGLE-1044 > Project: Eagle > Issue Type: New Feature >Affects Versions: v0.5.1 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Critical > > This new feature is to facilitate policy management among multiple sites. > With a single policy prototype, it can be imported into different sites. > * add a new entity PolicyEntity in the metadata database > * provides RESTful APIs to query/delete/update a policy template > * easy to onboard policies on a new site -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1038) Support alertDuplication customization for each policy
[ https://issues.apache.org/jira/browse/EAGLE-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1038: - Affects Version/s: (was: v0.6.0) v0.5.1 > Support alertDuplication customization for each policy > --- > > Key: EAGLE-1038 > URL: https://issues.apache.org/jira/browse/EAGLE-1038 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.1 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > > Requirements > * compatible with old versions > * enable alertDuplication check for each policy > * optimize DefaultDeduplicator -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1041) Support policy processing pipeline
[
https://issues.apache.org/jira/browse/EAGLE-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1041:
-
Affects Version/s: (was: v0.5.0)
v0.5.1
> Support policy processing pipeline
> --
>
> Key: EAGLE-1041
> URL: https://issues.apache.org/jira/browse/EAGLE-1041
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.5.1
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
>
> In some cases, like increment or decrement pattern, data need to be processed
> in more than one stage. For example, alert if the metric value increases by
> N. Two steps to get the right alert.
> 1. sort & filter events which meet the filter conditions.
> 2. define data change pattern.
> Here is a sample policy
> {code}
> fromHADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"]#window.externalTime(timestamp,1min)
> select * group by site,host,component, metric insert into temp;
> from every
> a=HADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"],
> b=HADOOP_JMX_METRIC_STREAM_SANDBOX[b.component==a.component and
> b.metric==a.metric and b.host==a.host and b.value>a.value and a.value>100]
> select b.site,b.host,b.component, b.metric, b.value as newNumOfMissingBlocks,
> a.value as oldNumOfMissingBlocks, (b.value-a.value) as
> increastedNumOfMissingBlocks insert into
> HADOOP_JMX_METRIC_STREAM_SANDBOX_MISS_BLOCKS_LARGER_OUT;
> {code}
> There are two queries in this policy. The first one with the time window
> condition tells Eagle to sort the original events. The second one defines the
> data pattern. As the constraint of Siddhi syntax
> (https://docs.wso2.com/display/CEP420/SiddhiQL+Guide+3.1#SiddhiQLGuide3.1-Pattern),
> the filtering of input events does not work.
> Luckily, if we put the output stream of the first query as the input stream
> of the second query, it works. That's the problem this ticket tries to solve.
> Ideally, the right policy can be written as
> {code}
> fromHADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"]#window.externalTime(timestamp,1min)
> select * group by site,host,component, metric insert into MISSING_BLOCK_OUT;
> from every a=MISSING_BLOCK_OUT[metric=="hadoop.namenode.dfs.missingblocks"],
> b=MISSING_BLOCK_OUT[b.component==a.component and b.metric==a.metric and
> b.host==a.host and b.value>a.value and a.value>100]
> select b.site,b.host,b.component, b.metric, b.value as newNumOfMissingBlocks,
> a.value as oldNumOfMissingBlocks, (b.value-a.value) as
> increastedNumOfMissingBlocks insert into
> HADOOP_JMX_METRIC_STREAM_SANDBOX_MISS_BLOCKS_LARGER_OUT;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (EAGLE-1051) Check if a publisher can be deleted
[ https://issues.apache.org/jira/browse/EAGLE-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen resolved EAGLE-1051. -- Resolution: Fixed > Check if a publisher can be deleted > > > Key: EAGLE-1051 > URL: https://issues.apache.org/jira/browse/EAGLE-1051 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > > A publisher can be deleted only when there are no policies using it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1051) Check if a publisher can be deleted
[ https://issues.apache.org/jira/browse/EAGLE-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1051: - Fix Version/s: v0.5.0 > Check if a publisher can be deleted > > > Key: EAGLE-1051 > URL: https://issues.apache.org/jira/browse/EAGLE-1051 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.0 > > > A publisher can be deleted only when there are no policies using it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1055) Improve policy prototype apis
[
https://issues.apache.org/jira/browse/EAGLE-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1055:
-
Affects Version/s: (was: v0.6.0)
v0.5.1
Fix Version/s: v0.5.1
> Improve policy prototype apis
> --
>
> Key: EAGLE-1055
> URL: https://issues.apache.org/jira/browse/EAGLE-1055
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.5.1
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
> Fix For: v0.5.1
>
>
> 1. create a policy prototype from a policy
> {code}
> API: POST /rest/policyProto/create?needPolicyCreated=true
> Payload: PolicyEntity (policyDefinition + alertPublishmentIds)
> public class PolicyEntity {
> String name; // auto created
> PolicyDefinition definition;
> List alertPublishmentIds = new ArrayList<>();
> }
> {code}
> 2. create a policy prototype by policy name
> {code}
> API: POST /rest/policyProto/create/{policyId}
> {code}
> 3. create policies for site from a list of policy protoypes
> {code}
> API: POST /rest/policyProto/export/{site}
> Payload: List
> {code}
> 4. create policies for site from a list of prototypes
> {code}
> API: POST /rest/policyProto/exportByName/{site}
> Payload: List
> {code}
> 5. delete a prototype
> {code}
> API: DELETE /rest/policyProto/{uuid}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (EAGLE-1056) Fix a link bug in the email template
[ https://issues.apache.org/jira/browse/EAGLE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen resolved EAGLE-1056. -- Resolution: Fixed > Fix a link bug in the email template > > > Key: EAGLE-1056 > URL: https://issues.apache.org/jira/browse/EAGLE-1056 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Minor > Fix For: v0.5.1 > > > "View Alert Details" cannot be opened in the alert email. > The root cause is the uuid is inconsistent generated by different publisher > plugins -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1056) Fix a link bug in the email template
[ https://issues.apache.org/jira/browse/EAGLE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1056: - Description: "View Alert Details" cannot be opened in the alert email. The root cause is the uuid is inconsistent generated by different publisher plugins was: "View Alert Details" cannot be opened in the alert email. The root cause is the uuid is inconsistent by different publisher plugins > Fix a link bug in the email template > > > Key: EAGLE-1056 > URL: https://issues.apache.org/jira/browse/EAGLE-1056 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Minor > Fix For: v0.5.1 > > > "View Alert Details" cannot be opened in the alert email. > The root cause is the uuid is inconsistent generated by different publisher > plugins -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1056) Fix a link bug in the email template
[ https://issues.apache.org/jira/browse/EAGLE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1056: - Description: "View Alert Details" cannot be opened in the alert email. The root cause is the uuid is inconsistent by different publisher plugins was:"View Alert Details" cannot be opened in the alert email > Fix a link bug in the email template > > > Key: EAGLE-1056 > URL: https://issues.apache.org/jira/browse/EAGLE-1056 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Minor > Fix For: v0.5.1 > > > "View Alert Details" cannot be opened in the alert email. > The root cause is the uuid is inconsistent by different publisher plugins -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (EAGLE-1056) Fix a link bug in the email template
[ https://issues.apache.org/jira/browse/EAGLE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on EAGLE-1056 started by Zhao, Qingwen. > Fix a link bug in the email template > > > Key: EAGLE-1056 > URL: https://issues.apache.org/jira/browse/EAGLE-1056 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.1 > > > "View Alert Details" cannot be opened. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1056) Fix a link bug in the email template
[ https://issues.apache.org/jira/browse/EAGLE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1056: - Priority: Minor (was: Major) > Fix a link bug in the email template > > > Key: EAGLE-1056 > URL: https://issues.apache.org/jira/browse/EAGLE-1056 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Minor > Fix For: v0.5.1 > > > "View Alert Details" cannot be opened in the alert email -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1056) Fix a link bug in the email template
[ https://issues.apache.org/jira/browse/EAGLE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1056: - Affects Version/s: (was: v0.6.0) Fix Version/s: v0.5.1 > Fix a link bug in the email template > > > Key: EAGLE-1056 > URL: https://issues.apache.org/jira/browse/EAGLE-1056 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.1 > > > "View Alert Details" cannot be opened. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1056) Fix a link bug in the email template
[ https://issues.apache.org/jira/browse/EAGLE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1056: - Description: "View Alert Details" cannot be opened in the alert email (was: "View Alert Details" cannot be opened. ) > Fix a link bug in the email template > > > Key: EAGLE-1056 > URL: https://issues.apache.org/jira/browse/EAGLE-1056 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.1 > > > "View Alert Details" cannot be opened in the alert email -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1059) fix a bug in policy prototype resource
[ https://issues.apache.org/jira/browse/EAGLE-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1059: - Fix Version/s: v0.5.1 > fix a bug in policy prototype resource > -- > > Key: EAGLE-1059 > URL: https://issues.apache.org/jira/browse/EAGLE-1059 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.1 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.1 > > > When a PolicyEntity(or policy prototype) is exported as a policy on a site, > the partitionSpec of the new policy is not updated, which results in a wrong > policy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (EAGLE-1059) fix a bug in policy prototype resource
Zhao, Qingwen created EAGLE-1059: Summary: fix a bug in policy prototype resource Key: EAGLE-1059 URL: https://issues.apache.org/jira/browse/EAGLE-1059 Project: Eagle Issue Type: Bug Affects Versions: v0.5.1 Reporter: Zhao, Qingwen Assignee: Zhao, Qingwen When a PolicyEntity(or policy prototype) is exported as a policy on a site, the partitionSpec of the new policy is not updated, which results in a wrong policy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (EAGLE-1059) fix a bug in policy prototype resource
[ https://issues.apache.org/jira/browse/EAGLE-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on EAGLE-1059 started by Zhao, Qingwen. > fix a bug in policy prototype resource > -- > > Key: EAGLE-1059 > URL: https://issues.apache.org/jira/browse/EAGLE-1059 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.1 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.1 > > > When a PolicyEntity(or policy prototype) is exported as a policy on a site, > the partitionSpec of the new policy is not updated, which results in a wrong > policy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1055) Improve policy prototype apis
[
https://issues.apache.org/jira/browse/EAGLE-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1055:
-
Description:
1. create a policy prototype from a policy
{code}
API: POST /rest/policyProto/create?needPolicyCreated=true
Payload: PolicyEntity (policyDefinition + alertPublishmentIds)
public class PolicyEntity {
String name; // auto created
PolicyDefinition definition;
List alertPublishmentIds = new ArrayList<>();
}
{code}
2. create a policy prototype by policy name
{code}
API: POST /rest/policyProto/create/{policyId}
{code}
3. create policies for site from a list of policy protoypes
{code}
API: POST /rest/policyProto/export/{site}
Payload: List
{code}
4. create policies for site from a list of prototypes
{code}
API: POST /rest/policyProto/exportByName/{site}
Payload: List
{code}
5. delete a prototype
{code}
API: DELETE /rest/policyProto/{uuid}
{code}
was:
1. create a policy prototype from a policy
{code}
API: POST /rest/policyProto/create?needPolicyCreated=true
Payload: PolicyEntity (policyDefinition + alertPublishmentIds)
public class PolicyEntity extends PersistenceEntity {
String name; // auto created
PolicyDefinition definition;
List alertPublishmentIds = new ArrayList<>();
}
{code}
2. create a policy prototype by policy name
{code}
API: POST /rest/policyProto/create/{policyId}
{code}
3. create policies for site from a list of policy protoypes
{code}
API: POST /rest/policyProto/export/{site}
Payload: List
{code}
4. create policies for site from a list of prototypes
{code}
API: POST /rest/policyProto/exportByName/{site}
Payload: List
{code}
5. delete a prototype
{code}
API: DELETE /rest/policyProto/{uuid}
{code}
> Improve policy prototype apis
> --
>
> Key: EAGLE-1055
> URL: https://issues.apache.org/jira/browse/EAGLE-1055
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.6.0
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
>
> 1. create a policy prototype from a policy
> {code}
> API: POST /rest/policyProto/create?needPolicyCreated=true
> Payload: PolicyEntity (policyDefinition + alertPublishmentIds)
> public class PolicyEntity {
> String name; // auto created
> PolicyDefinition definition;
> List alertPublishmentIds = new ArrayList<>();
> }
> {code}
> 2. create a policy prototype by policy name
> {code}
> API: POST /rest/policyProto/create/{policyId}
> {code}
> 3. create policies for site from a list of policy protoypes
> {code}
> API: POST /rest/policyProto/export/{site}
> Payload: List
> {code}
> 4. create policies for site from a list of prototypes
> {code}
> API: POST /rest/policyProto/exportByName/{site}
> Payload: List
> {code}
> 5. delete a prototype
> {code}
> API: DELETE /rest/policyProto/{uuid}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (EAGLE-1055) Improve policy prototype apis
[
https://issues.apache.org/jira/browse/EAGLE-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1055:
-
Description:
1. create a policy prototype from a policy
{code}
API: POST /rest/policyProto/create?needPolicyCreated=true
Payload: PolicyEntity (policyDefinition + alertPublishmentIds)
public class PolicyEntity extends PersistenceEntity {
String name; // auto created
PolicyDefinition definition;
List alertPublishmentIds = new ArrayList<>();
}
{code}
2. create a policy prototype by policy name
{code}
API: POST /rest/policyProto/create/{policyId}
{code}
3. create policies for site from a list of policy protoypes
{code}
API: POST /rest/policyProto/export/{site}
Payload: List
{code}
4. create policies for site from a list of prototypes
{code}
API: POST /rest/policyProto/exportByName/{site}
Payload: List
{code}
5. delete a prototype
{code}
API: DELETE /rest/policyProto/{uuid}
{code}
was:
1. create a policy prototype from a policy
{code}
API: /rest/policyProto/create?needPolicyCreated=true
Payload: PolicyEntity (policy + alertPublishmentIds)
{code}
2. create a policy prototype by policy name
{code}
API: /rest/policyProto/create/\{policyId\}
{code}
3. create policies for site from a list of policy protoypes
{code}
API: /rest/policyProto/export/\{site\}
Payload: List
{code}
4. create policies for site from a list of prototypes
{code}
API: /rest/policyProto/exportByName/\{site\}
Payload: List
{code}
> Improve policy prototype apis
> --
>
> Key: EAGLE-1055
> URL: https://issues.apache.org/jira/browse/EAGLE-1055
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.6.0
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
>
> 1. create a policy prototype from a policy
> {code}
> API: POST /rest/policyProto/create?needPolicyCreated=true
> Payload: PolicyEntity (policyDefinition + alertPublishmentIds)
> public class PolicyEntity extends PersistenceEntity {
> String name; // auto created
> PolicyDefinition definition;
> List alertPublishmentIds = new ArrayList<>();
> }
> {code}
> 2. create a policy prototype by policy name
> {code}
> API: POST /rest/policyProto/create/{policyId}
> {code}
> 3. create policies for site from a list of policy protoypes
> {code}
> API: POST /rest/policyProto/export/{site}
> Payload: List
> {code}
> 4. create policies for site from a list of prototypes
> {code}
> API: POST /rest/policyProto/exportByName/{site}
> Payload: List
> {code}
> 5. delete a prototype
> {code}
> API: DELETE /rest/policyProto/{uuid}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (EAGLE-1044) Support policy import using a policy prototype
[ https://issues.apache.org/jira/browse/EAGLE-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1044: - Description: This new feature is to facilitate policy management among multiple sites. With a single policy prototype, it can be imported into different sites. * add a new entity PolicyEntity in the metadata database * provides RESTful APIs to query/delete/update a policy template * easy to onboard policies on a new site was: * add a new entity PolicyEntity in the metadata database * provides RESTful APIs to query/delete/update a policy template * easy to onboard policies on a new site > Support policy import using a policy prototype > -- > > Key: EAGLE-1044 > URL: https://issues.apache.org/jira/browse/EAGLE-1044 > Project: Eagle > Issue Type: New Feature >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Critical > > This new feature is to facilitate policy management among multiple sites. > With a single policy prototype, it can be imported into different sites. > * add a new entity PolicyEntity in the metadata database > * provides RESTful APIs to query/delete/update a policy template > * easy to onboard policies on a new site -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1044) Support policy import using a policy prototype
[ https://issues.apache.org/jira/browse/EAGLE-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1044: - Summary: Support policy import using a policy prototype (was: Support policy Import using a policy prototype) > Support policy import using a policy prototype > -- > > Key: EAGLE-1044 > URL: https://issues.apache.org/jira/browse/EAGLE-1044 > Project: Eagle > Issue Type: New Feature >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Critical > > * add a new entity PolicyEntity in the metadata database > * provides RESTful APIs to query/delete/update a policy template > * easy to onboard policies on a new site -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1056) Fix a link bug in the email template
[ https://issues.apache.org/jira/browse/EAGLE-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1056: - Summary: Fix a link bug in the email template (was: Fix a link bug in the Email template) > Fix a link bug in the email template > > > Key: EAGLE-1056 > URL: https://issues.apache.org/jira/browse/EAGLE-1056 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0, v0.6.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > > "View Alert Details" cannot be opened. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (EAGLE-1056) Fix a link bug in the Email template
Zhao, Qingwen created EAGLE-1056: Summary: Fix a link bug in the Email template Key: EAGLE-1056 URL: https://issues.apache.org/jira/browse/EAGLE-1056 Project: Eagle Issue Type: Bug Affects Versions: v0.5.0, v0.6.0 Reporter: Zhao, Qingwen Assignee: Zhao, Qingwen "View Alert Details" cannot be opened. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1055) Improve policy prototype apis
[
https://issues.apache.org/jira/browse/EAGLE-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1055:
-
Description:
1. create a policy prototype from a policy
{code}
API: /rest/policyProto/create?needPolicyCreated=true
Payload: PolicyEntity (policy + alertPublishmentIds)
{code}
2. create a policy prototype by policy name
{code}
API: /rest/policyProto/create/\{policyId\}
{code}
3. create policies for site from a list of policy protoypes
{code}
API: /rest/policyProto/export/\{site\}
Payload: List
{code}
4. create policies for site from a list of prototypes
{code}
API: /rest/policyProto/exportByName/\{site\}
Payload: List
{code}
was:
1. create a policy prototype from a policy
{code}
API: /rest/policyProto/create?needPolicyCreated=true
Payload: PolicyEntity (policy + alertPublishmentIds)
{code}
2. create a policy prototype by policy name
{code}
API: /rest/policyProto/create/\{policyId\}
{code}
3. create policies for {site} from a list of policy protoypes
{code}
API: /rest/policyProto/export/\{site\}
Payload: List
{code}
4. create policies for {site} from a list of prototypes
{code}
API: /rest/policyProto/exportByName/\{site\}
Payload: List
{code}
> Improve policy prototype apis
> --
>
> Key: EAGLE-1055
> URL: https://issues.apache.org/jira/browse/EAGLE-1055
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.6.0
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
>
> 1. create a policy prototype from a policy
> {code}
> API: /rest/policyProto/create?needPolicyCreated=true
> Payload: PolicyEntity (policy + alertPublishmentIds)
> {code}
> 2. create a policy prototype by policy name
> {code}
> API: /rest/policyProto/create/\{policyId\}
> {code}
> 3. create policies for site from a list of policy protoypes
> {code}
> API: /rest/policyProto/export/\{site\}
> Payload: List
> {code}
> 4. create policies for site from a list of prototypes
> {code}
> API: /rest/policyProto/exportByName/\{site\}
> Payload: List
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (EAGLE-1055) Improve policy prototype apis
[
https://issues.apache.org/jira/browse/EAGLE-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1055:
-
Description:
1. create a policy prototype from a policy
{code}
API: /rest/policyProto/create?needPolicyCreated=true
Payload: PolicyEntity (policy + alertPublishmentIds)
{code}
2. create a policy prototype by policy name
{code}
API: /rest/policyProto/create/\{policyId\}
{code}
3. create policies for {site} from a list of policy protoypes
{code}
API: /rest/policyProto/export/\{site\}
Payload: List
{code}
4. create policies for {site} from a list of prototypes
{code}
API: /rest/policyProto/exportByName/\{site\}
Payload: List
{code}
was:
1. create a policy prototype from a policy
API: /rest/policyProto/create?needPolicyCreated=true
Payload: PolicyEntity (policy + alertPublishmentIds)
2. create a policy prototype by policy name
API: /rest/policyProto/create/{policyId}
3. create policies for {site} from a list of policy protoypes
API: /rest/policyProto/export/{site}
Payload: List
4. create policies for {site} from a list of prototypes
API: /rest/policyProto/exportByName/{site}
Payload: List
> Improve policy prototype apis
> --
>
> Key: EAGLE-1055
> URL: https://issues.apache.org/jira/browse/EAGLE-1055
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.6.0
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
>
> 1. create a policy prototype from a policy
> {code}
> API: /rest/policyProto/create?needPolicyCreated=true
> Payload: PolicyEntity (policy + alertPublishmentIds)
> {code}
> 2. create a policy prototype by policy name
> {code}
> API: /rest/policyProto/create/\{policyId\}
> {code}
> 3. create policies for {site} from a list of policy protoypes
> {code}
> API: /rest/policyProto/export/\{site\}
> Payload: List
> {code}
> 4. create policies for {site} from a list of prototypes
> {code}
> API: /rest/policyProto/exportByName/\{site\}
> Payload: List
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (EAGLE-1055) Improve policy prototype apis
[
https://issues.apache.org/jira/browse/EAGLE-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1055:
-
Description:
1. create a policy prototype from a policy
API: /rest/policyProto/create?needPolicyCreated=true
Payload: PolicyEntity (policy + alertPublishmentIds)
2. create a policy prototype by policy name
API: /rest/policyProto/create/{policyId}
3. create policies for {site} from a list of policy protoypes
API: /rest/policyProto/export/{site}
Payload: List
4. create policies for {site} from a list of prototypes
API: /rest/policyProto/exportByName/{site}
Payload: List
was:
{{POST /rest/policyProto/create?needPolicyCreated=true}}
1. if needPolicyCreated = true, create a new policy definition
2. create a policy prototype
{{POST /rest/policyProto/create/{policyId}}}
1. create a policy prototype
> Improve policy prototype apis
> --
>
> Key: EAGLE-1055
> URL: https://issues.apache.org/jira/browse/EAGLE-1055
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.6.0
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
>
> 1. create a policy prototype from a policy
> API: /rest/policyProto/create?needPolicyCreated=true
> Payload: PolicyEntity (policy + alertPublishmentIds)
> 2. create a policy prototype by policy name
> API: /rest/policyProto/create/{policyId}
> 3. create policies for {site} from a list of policy protoypes
> API: /rest/policyProto/export/{site}
> Payload: List
> 4. create policies for {site} from a list of prototypes
> API: /rest/policyProto/exportByName/{site}
> Payload: List
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (EAGLE-1055) Improve policy prototype apis
Zhao, Qingwen created EAGLE-1055:
Summary: Improve policy prototype apis
Key: EAGLE-1055
URL: https://issues.apache.org/jira/browse/EAGLE-1055
Project: Eagle
Issue Type: Improvement
Affects Versions: v0.6.0
Reporter: Zhao, Qingwen
Assignee: Zhao, Qingwen
{{POST /rest/policyProto/create?needPolicyCreated=true}}
1. if needPolicyCreated = true, create a new policy definition
2. create a policy prototype
{{POST /rest/policyProto/create/{policyId}}}
1. create a policy prototype
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (EAGLE-1053) Support typeahead in Eagle UI
[ https://issues.apache.org/jira/browse/EAGLE-1053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1053: - Fix Version/s: v0.6.0 > Support typeahead in Eagle UI > - > > Key: EAGLE-1053 > URL: https://issues.apache.org/jira/browse/EAGLE-1053 > Project: Eagle > Issue Type: New Feature >Reporter: Zhao, Qingwen >Assignee: Jilin, Jiang > Fix For: v0.6.0 > > > Support typeahead in Eagle UI -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (EAGLE-1053) Support typeahead in Eagle UI
Zhao, Qingwen created EAGLE-1053: Summary: Support typeahead in Eagle UI Key: EAGLE-1053 URL: https://issues.apache.org/jira/browse/EAGLE-1053 Project: Eagle Issue Type: New Feature Reporter: Zhao, Qingwen Assignee: Jilin, Jiang Support typeahead in Eagle UI -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (EAGLE-1052) dump job list data into a csv file
Zhao, Qingwen created EAGLE-1052: Summary: dump job list data into a csv file Key: EAGLE-1052 URL: https://issues.apache.org/jira/browse/EAGLE-1052 Project: Eagle Issue Type: New Feature Affects Versions: v0.6.0 Reporter: Zhao, Qingwen Priority: Minor Provide a Restful API to dump job list data into a csv file -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1046) Eagle supports policies import to a new site from a policy prototype
[
https://issues.apache.org/jira/browse/EAGLE-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1046:
-
Description:
0. policy prototype entity
{code}
String name
PolicyDefinition definition
List alertPublishmentIds
{code}
1. load a list of policies to new site "sandbox" from policy prototypes by
{{POST /rest/policyProto/export/sandbox}}
{code}
[ { "definition": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
]
{code}
2. create a new policy prototype with an existing policy by {{ POST
/rest/policyProto/import}}
{code}
{ "definition": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM_SANDBOX[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
{code}
3. update or create a policy prototype by by {{ POST /rest/policyProto}}
{code}
{
"policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000 or
reduceOpsPerSecond > 1000 o
[jira] [Updated] (EAGLE-1046) Eagle supports policies import to a new site from a policy prototype
[
https://issues.apache.org/jira/browse/EAGLE-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1046:
-
Description:
0. policy prototype entity
{code}
String name
PolicyDefinition definition
List alertPublishmentIds
{code}
1. load policies to new site "sandbox" from policy prototypes by {{POST
/rest/policyProto/export/sandbox}}
{code}
[ { "definition": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
]
{code}
2. create a new policy prototype with an existing policy by {{ POST
/rest/policyProto/import}}
{code}
{ "definition": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM_SANDBOX[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
{code}
3. update or create a policy prototype by by {{ POST /rest/policyProto}}
{code}
{
"policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000 or
reduceOpsPerSecond > 1000 or avgOpsPe
[jira] [Updated] (EAGLE-1046) Eagle supports policies import to a new site from a policy prototype
[
https://issues.apache.org/jira/browse/EAGLE-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1046:
-
Description:
1. load policies to new site "sandbox" from policy prototypes by {{POST
/rest/policyProto/export/sandbox}}
{code}
[ { "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
]
{code}
2. create a new policy prototype with an existing policy by {{ POST
/rest/policyProto/import}}
{code}
{ "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM_SANDBOX[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
}
}
{code}
3. update or create a policy prototype by by {{ POST /rest/policyProto}}
{code}
{
"policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000 or
reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask > 50]
select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": nul
[jira] [Created] (EAGLE-1051) Check if a publisher can be deleted
Zhao, Qingwen created EAGLE-1051: Summary: Check if a publisher can be deleted Key: EAGLE-1051 URL: https://issues.apache.org/jira/browse/EAGLE-1051 Project: Eagle Issue Type: Sub-task Affects Versions: v0.5.0 Reporter: Zhao, Qingwen Assignee: Zhao, Qingwen A publisher can be deleted only when there are no policies using it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1048) Delete an alert publisher on Eagle UI
[
https://issues.apache.org/jira/browse/EAGLE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1048:
-
Description:
Administer can delete an alert publisher on Eagle UI
Request: DELETE /publishments/\{name\}
Response:
{code}
OpResult {
public int code = 200; // 200 = SUCCESS
public String message = "";
}
{code}
was:
Administer can delete an alert publisher on Eagle UI
Request: DELETE /publishments/{name}
Response:
{code}
OpResult {
public int code = 200; // 200 = SUCCESS
public String message = "";
}
{code}
> Delete an alert publisher on Eagle UI
> -
>
> Key: EAGLE-1048
> URL: https://issues.apache.org/jira/browse/EAGLE-1048
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.6.0
>Reporter: Zhao, Qingwen
>Assignee: Jilin, Jiang
>Priority: Minor
>
> Administer can delete an alert publisher on Eagle UI
> Request: DELETE /publishments/\{name\}
> Response:
> {code}
> OpResult {
> public int code = 200; // 200 = SUCCESS
> public String message = "";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (EAGLE-1048) Delete an alert publisher on Eagle UI
[
https://issues.apache.org/jira/browse/EAGLE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1048:
-
Description:
Administer can delete an alert publisher on Eagle UI
Request: DELETE /publishments/{name}
Response:
{code}
OpResult {
public int code = 200; // 200 = SUCCESS
public String message = "";
}
{code}
was:
Administer can delete an alert publisher on Eagle UI
Request: DELETE /publishments/{name}
Response:
{code}
OpResult {
public int code = 200;
public String message = "";
}
{code}
> Delete an alert publisher on Eagle UI
> -
>
> Key: EAGLE-1048
> URL: https://issues.apache.org/jira/browse/EAGLE-1048
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.6.0
>Reporter: Zhao, Qingwen
>Assignee: Jilin, Jiang
>Priority: Minor
>
> Administer can delete an alert publisher on Eagle UI
> Request: DELETE /publishments/{name}
> Response:
> {code}
> OpResult {
> public int code = 200; // 200 = SUCCESS
> public String message = "";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (EAGLE-1048) Delete an alert publisher on Eagle UI
[
https://issues.apache.org/jira/browse/EAGLE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1048:
-
Description:
Administer can delete an alert publisher on Eagle UI
Request: DELETE /publishments/{name}
Response:
{code}
OpResult {
public int code = 200;
public String message = "";
}
{code}
was:Administer can delete an alert publisher on Eagle UI
> Delete an alert publisher on Eagle UI
> -
>
> Key: EAGLE-1048
> URL: https://issues.apache.org/jira/browse/EAGLE-1048
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.6.0
>Reporter: Zhao, Qingwen
>Assignee: Jilin, Jiang
>Priority: Minor
>
> Administer can delete an alert publisher on Eagle UI
> Request: DELETE /publishments/{name}
> Response:
> {code}
> OpResult {
> public int code = 200;
> public String message = "";
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (EAGLE-1050) Resume Eagle Jenkins check
Zhao, Qingwen created EAGLE-1050: Summary: Resume Eagle Jenkins check Key: EAGLE-1050 URL: https://issues.apache.org/jira/browse/EAGLE-1050 Project: Eagle Issue Type: Task Reporter: Zhao, Qingwen Assignee: Michael Wu Eagle Jenkins check seems stopped. Please help to check it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1044) Support policy Import using a policy prototype
[ https://issues.apache.org/jira/browse/EAGLE-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1044: - Description: * add a new entity PolicyEntity in the metadata database * provides RESTful APIs to query/delete/update a policy template * easy to onboard policies on a new site was: * add a new entity PolicyTemplate in the metadata database * provides RESTful APIs to query/delete/update a policy template * easy to onboard policies on a new site > Support policy Import using a policy prototype > -- > > Key: EAGLE-1044 > URL: https://issues.apache.org/jira/browse/EAGLE-1044 > Project: Eagle > Issue Type: New Feature >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Critical > > * add a new entity PolicyEntity in the metadata database > * provides RESTful APIs to query/delete/update a policy template > * easy to onboard policies on a new site -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1044) Support policy Import using a policy prototype
[ https://issues.apache.org/jira/browse/EAGLE-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1044: - Summary: Support policy Import using a policy prototype (was: Create a new policy using a policy template) > Support policy Import using a policy prototype > -- > > Key: EAGLE-1044 > URL: https://issues.apache.org/jira/browse/EAGLE-1044 > Project: Eagle > Issue Type: New Feature >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Critical > > * add a new entity PolicyTemplate in the metadata database > * provides RESTful APIs to query/delete/update a policy template > * easy to onboard policies on a new site -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1046) Eagle supports policies import to a new site from a policy prototype
[
https://issues.apache.org/jira/browse/EAGLE-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1046:
-
Description:
1. load policies to new site "sandbox" from policy prototypes by {{POST
/rest/policyProto/loadToSite/sandbox}}
{code}
[ { "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
]
{code}
2. create a new policy prototype with an existing policy by {{ POST
/rest/policyProto/saveAsProto}}
{code}
{ "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM_SANDBOX[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
}
}
{code}
3. update or create a policy prototype by by {{ POST /rest/policyProto}}
{code}
{
"policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000 or
reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask > 50]
select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerCl
[jira] [Updated] (EAGLE-1046) Eagle supports policies import to a new site from a policy prototype
[
https://issues.apache.org/jira/browse/EAGLE-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1046:
-
Description:
1. load policies to new site "sandbox" from policy prototypes by {{POST
/rest/policyProto/loadToSite/sandbox}}
{code}
[ { "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
]
{code}
2. create a new policy prototype with an existing policy by {{ POST
/rest/policyProto/saveAsProto}}
{code}
{ "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM_SANDBOX[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
}
}
{code}
3. update or create a policy prototype by by {{ POST /rest/policyProto}}
{code}
{
"policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000 or
reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask > 50]
select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerCl
[jira] [Updated] (EAGLE-1046) Eagle supports policies import to a new site from a policy prototype
[
https://issues.apache.org/jira/browse/EAGLE-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1046:
-
Description:
1. load policies to new site "sandbox" from policy prototypes by {{POST
/rest/policyProto/loadToSite/sandbox}}
{code}
[ { "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
]
{code}
2. create a new policy prototype with an existing policy by {{ POST
/rest/policyProto/saveAsProto}}
{code}
{ "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM_SANDBOX[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
}
}
{code}
3. update or create a policy prototype by by {{ POST /rest/policyProto}}
{code}
{
"policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000 or
reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask > 50]
select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerCl
[jira] [Updated] (EAGLE-1046) Eagle supports policies import to a new site from a policy prototype
[
https://issues.apache.org/jira/browse/EAGLE-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1046:
-
Description:
1. load policies to new site "sandbox" from policy prototypes by {{POST
/rest/policyProto/loadToSite/sandbox}}
{code}
[ { "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
]
{code}
2. create a new policy prototype with an existing policy by {{ POST
/rest/policyProto/saveAsProto}}
{code}
{ "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM_SANDBOX[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
}
}
{code}
3. update or create a policy prototype by by {{ POST /rest/policyProto}}
{code}
{
"policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000 or
reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask > 50]
select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerCl
[jira] [Updated] (EAGLE-1046) Eagle supports policies import to a new site from a policy prototype
[
https://issues.apache.org/jira/browse/EAGLE-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1046:
-
Description:
1. load policies to new site "sandbox" from policy prototypes {{POST
/rest/policyProto/loadToSite/sandbox}}
{code}
[ { "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
]
{code}
2. create a new policy prototype with an existing policy
POST /rest/policyProto/saveAsProto
{code}
{ "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM_SANDBOX[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
}
}
{code}
3. update or create a policy prototype
{code}
{
"policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000 or
reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask > 50]
select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties"
[jira] [Created] (EAGLE-1049) Support metric filter in the metric preview page
Zhao, Qingwen created EAGLE-1049: Summary: Support metric filter in the metric preview page Key: EAGLE-1049 URL: https://issues.apache.org/jira/browse/EAGLE-1049 Project: Eagle Issue Type: Improvement Affects Versions: v0.6.0 Reporter: Zhao, Qingwen Assignee: Jilin, Jiang Priority: Minor Support metric filter in the metric preview page -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1048) Delete an alert publisher on Eagle UI
[ https://issues.apache.org/jira/browse/EAGLE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1048: - Priority: Minor (was: Major) > Delete an alert publisher on Eagle UI > - > > Key: EAGLE-1048 > URL: https://issues.apache.org/jira/browse/EAGLE-1048 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.6.0 >Reporter: Zhao, Qingwen >Assignee: Jilin, Jiang >Priority: Minor > > Administer can delete an alert publisher on Eagle UI -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (EAGLE-1048) Delete an alert publisher on Eagle UI
Zhao, Qingwen created EAGLE-1048: Summary: Delete an alert publisher on Eagle UI Key: EAGLE-1048 URL: https://issues.apache.org/jira/browse/EAGLE-1048 Project: Eagle Issue Type: Improvement Affects Versions: v0.6.0 Reporter: Zhao, Qingwen Assignee: Jilin, Jiang Administer can delete an alert publisher on Eagle UI -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1047) View all alert publishers on Eagle UI
[ https://issues.apache.org/jira/browse/EAGLE-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1047: - Description: Users can view all the publisher configurations > View all alert publishers on Eagle UI > - > > Key: EAGLE-1047 > URL: https://issues.apache.org/jira/browse/EAGLE-1047 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.6.0 >Reporter: Zhao, Qingwen >Assignee: Jilin, Jiang > > Users can view all the publisher configurations -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (EAGLE-1047) View all alert publishers on Eagle UI
Zhao, Qingwen created EAGLE-1047: Summary: View all alert publishers on Eagle UI Key: EAGLE-1047 URL: https://issues.apache.org/jira/browse/EAGLE-1047 Project: Eagle Issue Type: Improvement Affects Versions: v0.6.0 Reporter: Zhao, Qingwen Assignee: Jilin, Jiang -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1046) Eagle supports policies import to a new site from a policy prototype
[
https://issues.apache.org/jira/browse/EAGLE-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1046:
-
Description:
1. load policies to new site "sandbox" from policy prototypes `POST
/rest/policyProto/loadToSite/sandbox`
{code}
[ { "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
]
{code}
2. create a new policy prototype with an existing policy
POST /rest/policyProto/saveAsProto
{code}
{ "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM_SANDBOX[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
}
}
{code}
3. update or create a policy prototype
{code}
{
"policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000 or
reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask > 50]
select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties":
[jira] [Created] (EAGLE-1046) Eagle supports policies import to a new site from a policy prototype
Zhao, Qingwen created EAGLE-1046:
Summary: Eagle supports policies import to a new site from a
policy prototype
Key: EAGLE-1046
URL: https://issues.apache.org/jira/browse/EAGLE-1046
Project: Eagle
Issue Type: New Feature
Affects Versions: v0.6.0
Reporter: Zhao, Qingwen
Assignee: Jilin, Jiang
1. load policies to new site `sandbox`
POST /rest/policyProto/loadToSite/sandbox
{code}
[ { "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": null,
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
},
"alertPublishmentIds": []
}
]
{code}
2. save a policy prototype with an existing policy
POST /rest/policyProto/saveAsProto
{code}
{ "policyProto": {
"name": "JobRpcThroughput",
"description": "Policy for
MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"inputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX"
],
"outputStreams": [
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from MAP_REDUCE_JOB_STREAM_SANDBOX[mapOpsPerSecond > 1000
or reduceOpsPerSecond > 1000 or avgOpsPerMapTask > 50 or avgOpsPerReduceTask >
50] select * insert into MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "DISABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site job rpc",
"body": "$site job rpc throughput",
"severity": "WARNING",
"category": "JPM"
},
"alertDeduplications": [
{
"outputStreamId":
"MAP_REDUCE_JOB_STREAM_SANDBOX_RPC_THROUGHPUT_OUT",
"dedupIntervalMin": "0",
"dedupFields": []
}
],
"partitionSpec": [
{
"streamId": "MAP_REDUCE_JOB_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "JPM"
}
}
{code}
3. get all policy prototypes
GET /rest/policyProto
4. delete a policy prototype
DELETE /rest/policyProto/{uuid}
5. update a policy prototype
POST /rest/policyProto
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (EAGLE-1044) Create a new policy using a policy template
Zhao, Qingwen created EAGLE-1044: Summary: Create a new policy using a policy template Key: EAGLE-1044 URL: https://issues.apache.org/jira/browse/EAGLE-1044 Project: Eagle Issue Type: New Feature Reporter: Zhao, Qingwen Assignee: Zhao, Qingwen Priority: Critical * add a new entity PolicyTemplate in the metadata database * provides RESTful APIs to query/delete/update a policy template * easy to onboard policies on a new site -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (EAGLE-1041) Support policy processing pipeline
Zhao, Qingwen created EAGLE-1041:
Summary: Support policy processing pipeline
Key: EAGLE-1041
URL: https://issues.apache.org/jira/browse/EAGLE-1041
Project: Eagle
Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Zhao, Qingwen
Assignee: Zhao, Qingwen
In some cases, like increment or decrement pattern, data need to be processed
in more than one stage. For example, alert if the metric value increases by N.
Two steps to get the right alert.
1. sort & filter events which meet the filter conditions.
2. define data change pattern.
Here is a sample policy
{code}
fromHADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"]#window.externalTime(timestamp,1min)
select * group by site,host,component, metric insert into temp;
from every
a=HADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"],
b=HADOOP_JMX_METRIC_STREAM_SANDBOX[b.component==a.componentandb.metric==a.metricandb.host==a.hostandb.value>a.valueanda.value>100]
selectb.site,b.host,b.component, b.metric, b.value as newNumOfMissingBlocks,
a.value as oldNumOfMissingBlocks, (b.value-a.value) as
increastedNumOfMissingBlocks insert into
HADOOP_JMX_METRIC_STREAM_SANDBOX_MISS_BLOCKS_LARGER_OUT;
{code}
There are two queries in this policy. The first one with the time window
condition tells Eagle to sort the original events. The second one defines the
data pattern. As the constraint of Siddhi syntax
(https://docs.wso2.com/display/CEP420/SiddhiQL+Guide+3.1#SiddhiQLGuide3.1-Pattern),
the filtering of input events does not work.
Luckily, if we put the output stream of the first query as the input stream of
the second query, it works. That's the problem this ticket tries to solve.
Ideally, the right policy can be written as
{code}
fromHADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"]#window.externalTime(timestamp,1min)
select * group by site,host,component, metric insert into MISSING_BLOCK_OUT;
from every a=MISSING_BLOCK_OUT[metric=="hadoop.namenode.dfs.missingblocks"],
b=MISSING_BLOCK_OUT[b.component==a.componentandb.metric==a.metricandb.host==a.hostandb.value>a.valueanda.value>100]
selectb.site,b.host,b.component, b.metric, b.value as newNumOfMissingBlocks,
a.value as oldNumOfMissingBlocks, (b.value-a.value) as
increastedNumOfMissingBlocks insert into
HADOOP_JMX_METRIC_STREAM_SANDBOX_MISS_BLOCKS_LARGER_OUT;
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (EAGLE-1040) Metric
[ https://issues.apache.org/jira/browse/EAGLE-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen closed EAGLE-1040. Resolution: Not A Problem > Metric > -- > > Key: EAGLE-1040 > URL: https://issues.apache.org/jira/browse/EAGLE-1040 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen > > In the metric preview page(/#/metric/preview), users can choose host and value -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (EAGLE-1041) Support policy processing pipeline
[
https://issues.apache.org/jira/browse/EAGLE-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1041:
-
Description:
In some cases, like increment or decrement pattern, data need to be processed
in more than one stage. For example, alert if the metric value increases by N.
Two steps to get the right alert.
1. sort & filter events which meet the filter conditions.
2. define data change pattern.
Here is a sample policy
{code}
fromHADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"]#window.externalTime(timestamp,1min)
select * group by site,host,component, metric insert into temp;
from every
a=HADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"],
b=HADOOP_JMX_METRIC_STREAM_SANDBOX[b.component==a.component and
b.metric==a.metric and b.host==a.host and b.value>a.value and a.value>100]
select b.site,b.host,b.component, b.metric, b.value as newNumOfMissingBlocks,
a.value as oldNumOfMissingBlocks, (b.value-a.value) as
increastedNumOfMissingBlocks insert into
HADOOP_JMX_METRIC_STREAM_SANDBOX_MISS_BLOCKS_LARGER_OUT;
{code}
There are two queries in this policy. The first one with the time window
condition tells Eagle to sort the original events. The second one defines the
data pattern. As the constraint of Siddhi syntax
(https://docs.wso2.com/display/CEP420/SiddhiQL+Guide+3.1#SiddhiQLGuide3.1-Pattern),
the filtering of input events does not work.
Luckily, if we put the output stream of the first query as the input stream of
the second query, it works. That's the problem this ticket tries to solve.
Ideally, the right policy can be written as
{code}
fromHADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"]#window.externalTime(timestamp,1min)
select * group by site,host,component, metric insert into MISSING_BLOCK_OUT;
from every a=MISSING_BLOCK_OUT[metric=="hadoop.namenode.dfs.missingblocks"],
b=MISSING_BLOCK_OUT[b.component==a.component and b.metric==a.metric and
b.host==a.host and b.value>a.value and a.value>100]
select b.site,b.host,b.component, b.metric, b.value as newNumOfMissingBlocks,
a.value as oldNumOfMissingBlocks, (b.value-a.value) as
increastedNumOfMissingBlocks insert into
HADOOP_JMX_METRIC_STREAM_SANDBOX_MISS_BLOCKS_LARGER_OUT;
{code}
was:
In some cases, like increment or decrement pattern, data need to be processed
in more than one stage. For example, alert if the metric value increases by N.
Two steps to get the right alert.
1. sort & filter events which meet the filter conditions.
2. define data change pattern.
Here is a sample policy
{code}
fromHADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"]#window.externalTime(timestamp,1min)
select * group by site,host,component, metric insert into temp;
from every
a=HADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"],
b=HADOOP_JMX_METRIC_STREAM_SANDBOX[b.component==a.componentandb.metric==a.metricandb.host==a.hostandb.value>a.valueanda.value>100]
selectb.site,b.host,b.component, b.metric, b.value as newNumOfMissingBlocks,
a.value as oldNumOfMissingBlocks, (b.value-a.value) as
increastedNumOfMissingBlocks insert into
HADOOP_JMX_METRIC_STREAM_SANDBOX_MISS_BLOCKS_LARGER_OUT;
{code}
There are two queries in this policy. The first one with the time window
condition tells Eagle to sort the original events. The second one defines the
data pattern. As the constraint of Siddhi syntax
(https://docs.wso2.com/display/CEP420/SiddhiQL+Guide+3.1#SiddhiQLGuide3.1-Pattern),
the filtering of input events does not work.
Luckily, if we put the output stream of the first query as the input stream of
the second query, it works. That's the problem this ticket tries to solve.
Ideally, the right policy can be written as
{code}
fromHADOOP_JMX_METRIC_STREAM_SANDBOX[metric=="hadoop.namenode.dfs.missingblocks"]#window.externalTime(timestamp,1min)
select * group by site,host,component, metric insert into MISSING_BLOCK_OUT;
from every a=MISSING_BLOCK_OUT[metric=="hadoop.namenode.dfs.missingblocks"],
b=MISSING_BLOCK_OUT[b.component==a.componentandb.metric==a.metricandb.host==a.hostandb.value>a.valueanda.value>100]
selectb.site,b.host,b.component, b.metric, b.value as newNumOfMissingBlocks,
a.value as oldNumOfMissingBlocks, (b.value-a.value) as
increastedNumOfMissingBlocks insert into
HADOOP_JMX_METRIC_STREAM_SANDBOX_MISS_BLOCKS_LARGER_OUT;
{code}
> Support policy processing pipeline
> --
>
> Key: EAGLE-1041
> URL: https://issues.apache.org/jira/browse/EAGLE-1041
> Project: Eagle
> Issue Type: Improvement
>Affects Versions: v0.5.0
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
>
> In some cases, like increment or decrement pattern, data need to be processed
> in more than one stage. For example, alert if th
[jira] [Created] (EAGLE-1040) Metric
Zhao, Qingwen created EAGLE-1040: Summary: Metric Key: EAGLE-1040 URL: https://issues.apache.org/jira/browse/EAGLE-1040 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Zhao, Qingwen In the metric preview page(/#/metric/preview), users can choose host and value -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (EAGLE-1039) load alert de-duplication configuration in the page edit page
Zhao, Qingwen created EAGLE-1039: Summary: load alert de-duplication configuration in the page edit page Key: EAGLE-1039 URL: https://issues.apache.org/jira/browse/EAGLE-1039 Project: Eagle Issue Type: Improvement Affects Versions: v0.6.0 Reporter: Zhao, Qingwen Assignee: Jilin, Jiang If a user wants to edit an existing policy, alert de-duplication info should be loaded from the database when he/she enters to that page. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (EAGLE-1038) Support alertDuplication customization for each policy
[ https://issues.apache.org/jira/browse/EAGLE-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1038: - Description: Requirements * compatible with old versions * enable alertDuplication check for each policy * optimize DefaultDeduplicator was: Requirements * compatible with old versions * enable alertDuplication check for each policy > Support alertDuplication customization for each policy > --- > > Key: EAGLE-1038 > URL: https://issues.apache.org/jira/browse/EAGLE-1038 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.6.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > > Requirements > * compatible with old versions > * enable alertDuplication check for each policy > * optimize DefaultDeduplicator -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (EAGLE-1038) Support alertDuplication customization for each policy
[ https://issues.apache.org/jira/browse/EAGLE-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1038: - Summary: Support alertDuplication customization for each policy (was: enable alertDuplication check for each policy ) > Support alertDuplication customization for each policy > --- > > Key: EAGLE-1038 > URL: https://issues.apache.org/jira/browse/EAGLE-1038 > Project: Eagle > Issue Type: Sub-task >Affects Versions: v0.6.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > > Requirements > * compatible with old versions > * enable alertDuplication check for each policy -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (EAGLE-1037) Add alertDeduplication configurations on Eagle UI
[
https://issues.apache.org/jira/browse/EAGLE-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1037:
-
Description:
Add alertDeduplication configurations on Eagle UI
Here is the sample policy
{code}
{
"name": "capacityUsage",
"description": "Policy for
HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT",
"inputStreams": [
"HADOOP_JMX_METRIC_STREAM_SANDBOX"
],
"outputStreams": [
"HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from HADOOP_JMX_METRIC_STREAM_SANDBOX[metric ==
\"hadoop.namenode.fsnamesystemstate.capacityusage\" and convert(value,
\"long\") > 90]select site, host, component, metric, convert(value, \"long\")
as value, timestamp insert into
HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "ENABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site capacity exceeds 90%",
"body": "$site capacity exceeds 90%",
"severity": "WARNING",
"category": "HDFS"
},
"alertDeduplications": [
{
"outputStreamId": "HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT",
"dedupIntervalMin": "1",
"dedupFields": [
"site",
"component",
"host",
"metric"
]
}
],
"partitionSpec": [
{
"streamId": "HADOOP_JMX_METRIC_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "HDFS"
}
{code}
was:
Add alertDeduplication configurations on Eagle UI
Here is the Sample
{code}
{
"name": "capacityUsage",
"description": "Policy for
HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT",
"inputStreams": [
"HADOOP_JMX_METRIC_STREAM_SANDBOX"
],
"outputStreams": [
"HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from HADOOP_JMX_METRIC_STREAM_SANDBOX[metric ==
\"hadoop.namenode.fsnamesystemstate.capacityusage\" and convert(value,
\"long\") > 90]select site, host, component, metric, convert(value, \"long\")
as value, timestamp insert into
HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "ENABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site capacity exceeds 90%",
"body": "$site capacity exceeds 90%",
"severity": "WARNING",
"category": "HDFS"
},
"alertDeduplications": [
{
"outputStreamId": "HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT",
"dedupIntervalMin": "1",
"dedupFields": [
"site",
"component",
"host",
"metric"
]
}
],
"partitionSpec": [
{
"streamId": "HADOOP_JMX_METRIC_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "HDFS"
}
{code}
> Add alertDeduplication configurations on Eagle UI
> -
>
> Key: EAGLE-1037
> URL: https://issues.apache.org/jira/browse/EAGLE-1037
> Project: Eagle
> Issue Type: Sub-task
>Affects Versions: v0.6.0
>Reporter: Zhao, Qingwen
>Assignee: Jilin, Jiang
>Priority: Critical
>
> Add alertDeduplication configurations on Eagle UI
> Here is the sample policy
> {code}
> {
> "name": "capacityUsage",
> "description": "Policy for
> HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT",
> "inputStreams": [
> "HADOOP_JMX_METRIC_STREAM_SANDBOX"
> ],
> "outputStreams": [
> "HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT"
> ],
> "siteId": "sandbox",
> "definition": {
> "type": "siddhi",
> "value": "from HADOOP_JMX_METRIC_STREAM_SANDBOX[metric ==
> \"hadoop.namenode.fsnamesystemstate.capacityusage\" and convert(value,
> \"long\") > 90]select site, host, component, metric, convert(value, \"long\")
> as value, timestamp insert into
> HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT;",
> "handlerClass": null,
> "properties": {},
> "inputStreams": [],
> "outputStreams": []
> },
> "stateDefinition": null,
> "policyStatus": "ENABLED",
> "alertDefinition": {
>
[jira] [Updated] (EAGLE-1037) Add alertDeduplication configurations on Eagle UI
[
https://issues.apache.org/jira/browse/EAGLE-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen updated EAGLE-1037:
-
Description:
Add alertDeduplication configurations on Eagle UI
Here is the Sample
{code}
{
"name": "capacityUsage",
"description": "Policy for
HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT",
"inputStreams": [
"HADOOP_JMX_METRIC_STREAM_SANDBOX"
],
"outputStreams": [
"HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT"
],
"siteId": "sandbox",
"definition": {
"type": "siddhi",
"value": "from HADOOP_JMX_METRIC_STREAM_SANDBOX[metric ==
\"hadoop.namenode.fsnamesystemstate.capacityusage\" and convert(value,
\"long\") > 90]select site, host, component, metric, convert(value, \"long\")
as value, timestamp insert into
HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT;",
"handlerClass": null,
"properties": {},
"inputStreams": [],
"outputStreams": []
},
"stateDefinition": null,
"policyStatus": "ENABLED",
"alertDefinition": {
"templateType": "TEXT",
"subject": "$site capacity exceeds 90%",
"body": "$site capacity exceeds 90%",
"severity": "WARNING",
"category": "HDFS"
},
"alertDeduplications": [
{
"outputStreamId": "HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT",
"dedupIntervalMin": "1",
"dedupFields": [
"site",
"component",
"host",
"metric"
]
}
],
"partitionSpec": [
{
"streamId": "HADOOP_JMX_METRIC_STREAM_SANDBOX",
"type": "SHUFFLE",
"columns": [],
"sortSpec": null
}
],
"dedicated": false,
"parallelismHint": 5,
"alertSeverity": "WARNING",
"alertCategory": "HDFS"
}
{code}
was:Add alertDeduplication configurations on Eagle UI
> Add alertDeduplication configurations on Eagle UI
> -
>
> Key: EAGLE-1037
> URL: https://issues.apache.org/jira/browse/EAGLE-1037
> Project: Eagle
> Issue Type: Sub-task
>Affects Versions: v0.6.0
>Reporter: Zhao, Qingwen
>Assignee: Jilin, Jiang
>Priority: Critical
>
> Add alertDeduplication configurations on Eagle UI
> Here is the Sample
> {code}
> {
> "name": "capacityUsage",
> "description": "Policy for
> HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT",
> "inputStreams": [
> "HADOOP_JMX_METRIC_STREAM_SANDBOX"
> ],
> "outputStreams": [
> "HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT"
> ],
> "siteId": "sandbox",
> "definition": {
> "type": "siddhi",
> "value": "from HADOOP_JMX_METRIC_STREAM_SANDBOX[metric ==
> \"hadoop.namenode.fsnamesystemstate.capacityusage\" and convert(value,
> \"long\") > 90]select site, host, component, metric, convert(value, \"long\")
> as value, timestamp insert into
> HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT;",
> "handlerClass": null,
> "properties": {},
> "inputStreams": [],
> "outputStreams": []
> },
> "stateDefinition": null,
> "policyStatus": "ENABLED",
> "alertDefinition": {
> "templateType": "TEXT",
> "subject": "$site capacity exceeds 90%",
> "body": "$site capacity exceeds 90%",
> "severity": "WARNING",
> "category": "HDFS"
> },
> "alertDeduplications": [
> {
> "outputStreamId":
> "HADOOP_JMX_METRIC_STREAM_SANDBOX_CAPACITY_USAGE_OUT",
> "dedupIntervalMin": "1",
> "dedupFields": [
> "site",
> "component",
> "host",
> "metric"
> ]
> }
> ],
> "partitionSpec": [
> {
> "streamId": "HADOOP_JMX_METRIC_STREAM_SANDBOX",
> "type": "SHUFFLE",
> "columns": [],
> "sortSpec": null
> }
> ],
> "dedicated": false,
> "parallelismHint": 5,
> "alertSeverity": "WARNING",
> "alertCategory": "HDFS"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (EAGLE-1038) enable alertDuplication check for each policy
Zhao, Qingwen created EAGLE-1038: Summary: enable alertDuplication check for each policy Key: EAGLE-1038 URL: https://issues.apache.org/jira/browse/EAGLE-1038 Project: Eagle Issue Type: Sub-task Affects Versions: v0.6.0 Reporter: Zhao, Qingwen Assignee: Zhao, Qingwen Requirements * compatible with old versions * enable alertDuplication check for each policy -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (EAGLE-1037) Add alertDeduplication configurations on Eagle UI
Zhao, Qingwen created EAGLE-1037: Summary: Add alertDeduplication configurations on Eagle UI Key: EAGLE-1037 URL: https://issues.apache.org/jira/browse/EAGLE-1037 Project: Eagle Issue Type: Sub-task Affects Versions: v0.6.0 Reporter: Zhao, Qingwen Assignee: Jilin, Jiang Priority: Critical Add alertDeduplication configurations on Eagle UI -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (EAGLE-1036) policies with deduplication settings
Zhao, Qingwen created EAGLE-1036: Summary: policies with deduplication settings Key: EAGLE-1036 URL: https://issues.apache.org/jira/browse/EAGLE-1036 Project: Eagle Issue Type: New Feature Affects Versions: v0.6.0 Reporter: Zhao, Qingwen This improvement moves de-duplication logic from alert publishers to the policy side. Users can set de-duplication interval and fields when creating a new policy. At the same time, it's compatible with previous versions -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (EAGLE-1032) Add policy duplication settings in the policy definition page
Zhao, Qingwen created EAGLE-1032: Summary: Add policy duplication settings in the policy definition page Key: EAGLE-1032 URL: https://issues.apache.org/jira/browse/EAGLE-1032 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Zhao, Qingwen Add policy duplication settings in the policy definition page. 1. FieldName: dedupIntervalMin Type: string (text) e.g., 1, 2, 10 2. FieldName: dedupFieldsType: array (checkbox) Description: show all stream columns of STRING type e.g. ["site", "component"] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (EAGLE-1031) add intervalMin setting in the metric preview page
Zhao, Qingwen created EAGLE-1031:
Summary: add intervalMin setting in the metric preview page
Key: EAGLE-1031
URL: https://issues.apache.org/jira/browse/EAGLE-1031
Project: Eagle
Issue Type: Improvement
Affects Versions: v0.5.0
Reporter: Zhao, Qingwen
Assignee: Zhao, Qingwen
By default, the intervalMin is calculated by
{code}
intervalMin = 5min, |queryEndTime - queryStartTime| <= 6h
15min, |queryEndTime - queryStartTime| <= 24h
30min, |queryEndTime - queryStartTime| <= 7d
60min, |queryEndTime - queryStartTime| <= 14d
1day, |queryEndTime - queryStartTime| > 14d
{code}
In some cases, users want to view metric in minute level. Eagle should support
to customize this parameter.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (EAGLE-1029) Wrong groupSpec is generated when more than one alert engine topology
[ https://issues.apache.org/jira/browse/EAGLE-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1029: - Description: According to the schedule info returned by /rest/metadata/schedulestates In alertSpecs, new policies are assigned to alertBolt 9, 18, 19, 1, 12 of the second alert engine topology. While in groupSpecs, the worker queue(alert bolts) allocated for each policy is not [ 9, 18, 19, 1, 12]. Actually, it adds all working queues, including those of other alert engine topology was: According to the schedule info returned by /rest/metadata/schedulestates In alertSpecs, new policies are assigned to alertBolt 9, 18, 19, 1, 12 of the second alert engine topology. While in groupSpecs, the worker queue(alert bolts) allocated for each policy is not [ 9, 18, 19, 1, 12] > Wrong groupSpec is generated when more than one alert engine topology > -- > > Key: EAGLE-1029 > URL: https://issues.apache.org/jira/browse/EAGLE-1029 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen >Priority: Critical > > According to the schedule info returned by /rest/metadata/schedulestates > In alertSpecs, new policies are assigned to alertBolt 9, 18, 19, 1, 12 of the > second alert engine topology. While in groupSpecs, the worker queue(alert > bolts) allocated for each policy is not [ 9, 18, 19, 1, 12]. Actually, it > adds all working queues, including those of other alert engine topology -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (EAGLE-1029) Wrong groupSpec is generated when more than one alert engine topology
Zhao, Qingwen created EAGLE-1029: Summary: Wrong groupSpec is generated when more than one alert engine topology Key: EAGLE-1029 URL: https://issues.apache.org/jira/browse/EAGLE-1029 Project: Eagle Issue Type: Bug Affects Versions: v0.5.0 Reporter: Zhao, Qingwen Assignee: Zhao, Qingwen Priority: Critical According to the schedule info returned by /rest/metadata/schedulestates In alertSpecs, new policies are assigned to alertBolt 9, 18, 19, 1, 12 of the second alert engine topology. While in groupSpecs, the worker queue(alert bolts) allocated for each policy is not [ 9, 18, 19, 1, 12] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (EAGLE-1024) Monitor jobs with high RPC throughput
[ https://issues.apache.org/jira/browse/EAGLE-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen resolved EAGLE-1024. -- Resolution: Done > Monitor jobs with high RPC throughput > -- > > Key: EAGLE-1024 > URL: https://issues.apache.org/jira/browse/EAGLE-1024 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > > We've identified some jobs with high RPC throughput which causes the NN heavy > RPC overhead. These jobs has requested extremely large HDFS operations in a > very short window (2 mins). > So we tend to capture those jobs with: > a) the job has very large RPC throughput, using the job total HDFS ops/the > job duration, if the throughput is larger than 1000 > b) and if the HDFS ops per task is larger than 25 > Then send out the alert out. Later, we will notify the users to optimize > their jobs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (EAGLE-1024) Monitor jobs with high RPC throughput
[ https://issues.apache.org/jira/browse/EAGLE-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1024: - Fix Version/s: v0.5.0 > Monitor jobs with high RPC throughput > -- > > Key: EAGLE-1024 > URL: https://issues.apache.org/jira/browse/EAGLE-1024 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.0 > > > We've identified some jobs with high RPC throughput which causes the NN heavy > RPC overhead. These jobs has requested extremely large HDFS operations in a > very short window (2 mins). > So we tend to capture those jobs with: > a) the job has very large RPC throughput, using the job total HDFS ops/the > job duration, if the throughput is larger than 1000 > b) and if the HDFS ops per task is larger than 25 > Then send out the alert out. Later, we will notify the users to optimize > their jobs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (EAGLE-1022) Configuration parsing exception in SparkHistoryJob app
[
https://issues.apache.org/jira/browse/EAGLE-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao, Qingwen resolved EAGLE-1022.
--
Resolution: Fixed
Fix Version/s: v0.5.0
> Configuration parsing exception in SparkHistoryJob app
> --
>
> Key: EAGLE-1022
> URL: https://issues.apache.org/jira/browse/EAGLE-1022
> Project: Eagle
> Issue Type: Bug
>Affects Versions: v0.5.0
>Reporter: Zhao, Qingwen
>Assignee: Zhao, Qingwen
> Fix For: v0.5.0
>
>
> {code}
> java.lang.NumberFormatException: For input string: "2g"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> ~[na:1.8.0_91]
> at java.lang.Integer.parseInt(Integer.java:580) ~[na:1.8.0_91]
> at java.lang.Integer.parseInt(Integer.java:615) ~[na:1.8.0_91]
> at org.apache.eagle.jpm.util.Utils.parseMemory(Utils.java:78)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.spark.history.crawl.JHFSparkEventReader.getMemoryOverhead(JHFSparkEventReader.java:490)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.spark.history.crawl.JHFSparkEventReader.clearReader(JHFSparkEventReader.java:449)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.spark.history.crawl.JHFSparkParser.parse(JHFSparkParser.java:53)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.spark.history.crawl.SparkFilesystemInputStreamReaderImpl.read(SparkFilesystemInputStreamReaderImpl.java:47)
> ~[stormjar.jar:na]
> at
> org.apache.eagle.jpm.spark.history.storm.SparkHistoryJobParseBolt.execute(SparkHistoryJobParseBolt.java:98)
> ~[stormjar.jar:na]
> at
> backtype.storm.daemon.executor$fn__5697$tuple_action_fn__5699.invoke(executor.clj:659)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at
> backtype.storm.daemon.executor$mk_task_receiver$fn__5620.invoke(executor.clj:415)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at
> backtype.storm.disruptor$clojure_handler$reify__1741.onEvent(disruptor.clj:58)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at
> backtype.storm.daemon.executor$fn__5697$fn__5710$fn__5761.invoke(executor.clj:794)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at backtype.storm.util$async_loop$fn__452.invoke(util.clj:465)
> [storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (EAGLE-1024) Monitor jobs with high RPC throughput
[ https://issues.apache.org/jira/browse/EAGLE-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen reassigned EAGLE-1024: Assignee: Zhao, Qingwen > Monitor jobs with high RPC throughput > -- > > Key: EAGLE-1024 > URL: https://issues.apache.org/jira/browse/EAGLE-1024 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > > We've identified some jobs with high RPC throughput which causes the NN heavy > RPC overhead. These jobs has requested extremely large HDFS operations in a > very short window (2 mins). > So we tend to capture those jobs with: > a) the job has very large RPC throughput, using the job total HDFS ops/the > job duration, if the throughput is larger than 1000 > b) and if the HDFS ops per task is larger than 25 > Then send out the alert out. Later, we will notify the users to optimize > their jobs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (EAGLE-1024) Monitor jobs with high RPC throughput
Zhao, Qingwen created EAGLE-1024: Summary: Monitor jobs with high RPC throughput Key: EAGLE-1024 URL: https://issues.apache.org/jira/browse/EAGLE-1024 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Zhao, Qingwen We've identified some jobs with high RPC throughput which causes the NN heavy RPC overhead. These jobs has requested extremely large HDFS operations in a very short window (2 mins). So we tend to capture those jobs with: a) the job has very large RPC throughput, using the job total HDFS ops/the job duration, if the throughput is larger than 1000 b) and if the HDFS ops per task is larger than 25 Then send out the alert out. Later, we will notify the users to optimize their jobs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (EAGLE-1023) update Hadoop jmx metric collector scripts & fix bugs
Zhao, Qingwen created EAGLE-1023: Summary: update Hadoop jmx metric collector scripts & fix bugs Key: EAGLE-1023 URL: https://issues.apache.org/jira/browse/EAGLE-1023 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Zhao, Qingwen Assignee: Zhao, Qingwen update Hadoop jmx metric collector scripts -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (EAGLE-1022) Configuration parsing exception in SparkHistoryJob app
Zhao, Qingwen created EAGLE-1022:
Summary: Configuration parsing exception in SparkHistoryJob app
Key: EAGLE-1022
URL: https://issues.apache.org/jira/browse/EAGLE-1022
Project: Eagle
Issue Type: Bug
Affects Versions: v0.5.0
Reporter: Zhao, Qingwen
Assignee: Zhao, Qingwen
{code}
java.lang.NumberFormatException: For input string: "2g"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
~[na:1.8.0_91]
at java.lang.Integer.parseInt(Integer.java:580) ~[na:1.8.0_91]
at java.lang.Integer.parseInt(Integer.java:615) ~[na:1.8.0_91]
at org.apache.eagle.jpm.util.Utils.parseMemory(Utils.java:78)
~[stormjar.jar:na]
at
org.apache.eagle.jpm.spark.history.crawl.JHFSparkEventReader.getMemoryOverhead(JHFSparkEventReader.java:490)
~[stormjar.jar:na]
at
org.apache.eagle.jpm.spark.history.crawl.JHFSparkEventReader.clearReader(JHFSparkEventReader.java:449)
~[stormjar.jar:na]
at
org.apache.eagle.jpm.spark.history.crawl.JHFSparkParser.parse(JHFSparkParser.java:53)
~[stormjar.jar:na]
at
org.apache.eagle.jpm.spark.history.crawl.SparkFilesystemInputStreamReaderImpl.read(SparkFilesystemInputStreamReaderImpl.java:47)
~[stormjar.jar:na]
at
org.apache.eagle.jpm.spark.history.storm.SparkHistoryJobParseBolt.execute(SparkHistoryJobParseBolt.java:98)
~[stormjar.jar:na]
at
backtype.storm.daemon.executor$fn__5697$tuple_action_fn__5699.invoke(executor.clj:659)
[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
at
backtype.storm.daemon.executor$mk_task_receiver$fn__5620.invoke(executor.clj:415)
[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
at
backtype.storm.disruptor$clojure_handler$reify__1741.onEvent(disruptor.clj:58)
[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
at
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
at
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
at
backtype.storm.daemon.executor$fn__5697$fn__5710$fn__5761.invoke(executor.clj:794)
[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
at backtype.storm.util$async_loop$fn__452.invoke(util.clj:465)
[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (EAGLE-432) Application status monitoring
[ https://issues.apache.org/jira/browse/EAGLE-432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen resolved EAGLE-432. - Resolution: Done > Application status monitoring > - > > Key: EAGLE-432 > URL: https://issues.apache.org/jira/browse/EAGLE-432 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Edward Zhang >Assignee: DanielZhou > Fix For: v0.5.0 > > > App framework needs to probe application status periodically or on demand. > ApplicationEntity contains status field, and this field's value should come > from physical storm applications. > We can start background threads to check application status or on-demand. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (EAGLE-964) Add a column 'severity' left to description in policy list page
[ https://issues.apache.org/jira/browse/EAGLE-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen resolved EAGLE-964. - Resolution: Fixed > Add a column 'severity' left to description in policy list page > --- > > Key: EAGLE-964 > URL: https://issues.apache.org/jira/browse/EAGLE-964 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.0 > > > Add a column 'severity' left to description in policy list page -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (EAGLE-964) Add a column 'severity' left to description in policy list page
[ https://issues.apache.org/jira/browse/EAGLE-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-964: Fix Version/s: v0.5.0 > Add a column 'severity' left to description in policy list page > --- > > Key: EAGLE-964 > URL: https://issues.apache.org/jira/browse/EAGLE-964 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.0 > > > Add a column 'severity' left to description in policy list page -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (EAGLE-1006) Fix memory leak issue in eagle server
[ https://issues.apache.org/jira/browse/EAGLE-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009949#comment-16009949 ] Zhao, Qingwen commented on EAGLE-1006: -- I am not sure all the issue have been found. However, after fixing the bug in ExecutionRuntimeManager, the server runs much more smoothly. > Fix memory leak issue in eagle server > - > > Key: EAGLE-1006 > URL: https://issues.apache.org/jira/browse/EAGLE-1006 > Project: Eagle > Issue Type: Bug >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.0 > > > After eagle server started for a long time, the response time will be very > slow to user request and the memory usage is very large. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (EAGLE-1006) Fix memory leak issue in eagle server
[ https://issues.apache.org/jira/browse/EAGLE-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen updated EAGLE-1006: - Summary: Fix memory leak issue in eagle server (was: Memory leak in Eagle Server) > Fix memory leak issue in eagle server > - > > Key: EAGLE-1006 > URL: https://issues.apache.org/jira/browse/EAGLE-1006 > Project: Eagle > Issue Type: Bug >Reporter: Zhao, Qingwen >Assignee: Zhao, Qingwen > Fix For: v0.5.0 > > > After eagle server started for a long time, the response time will be very > slow to user request and the memory usage is very large. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (EAGLE-1016) Mysql table eagle_metric_eagle_metric_schema is having issue
[ https://issues.apache.org/jira/browse/EAGLE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhao, Qingwen reassigned EAGLE-1016: Assignee: Zhao, Qingwen > Mysql table eagle_metric_eagle_metric_schema is having issue > > > Key: EAGLE-1016 > URL: https://issues.apache.org/jira/browse/EAGLE-1016 > Project: Eagle > Issue Type: Bug >Reporter: vikash kumar >Assignee: Zhao, Qingwen > Attachments: Apache Eagle Error log.txt > > > Hi, > eagle_metric_eagle_metric_schema is having a column_name as "group" which is > a reserved keyword. I am getting an error as mysql syntax error near group > when i am trying to run the apache eagle application for > hadoop_metric_monitor. Initially i was getting error even in creating this > table with group keyword as column but i resolved it using group .Now it is > trying to write to some of the table which is having group as a cloumn_name. > That's why it is throwing error. Can somebody help me out. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
