[jira] [Created] (EAGLE-837) Stream definition change does not reflect in AlertBolt
Garrett Li created EAGLE-837: Summary: Stream definition change does not reflect in AlertBolt Key: EAGLE-837 URL: https://issues.apache.org/jira/browse/EAGLE-837 Project: Eagle Issue Type: Bug Reporter: Garrett Li Assignee: Garrett Li Stream definition change only trigger router bolt & publisher update, we don't update corresponding alert bolt stream definition references. It will cause alert bolt still use old stream definition references, it could produce array index out of bound exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-824) Multiple policies in one alert bolt produces duplicated tuples to publisher
Garrett Li created EAGLE-824: Summary: Multiple policies in one alert bolt produces duplicated tuples to publisher Key: EAGLE-824 URL: https://issues.apache.org/jira/browse/EAGLE-824 Project: Eagle Issue Type: Bug Reporter: Garrett Li Assignee: Garrett Li Multiple policies in one alert bolt will cause each policy in this bolt produce the tuple and emit tuple into publisher, the publisher will got multiple duplicated tuples. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-794) Enable publish bolt parallelism
Garrett Li created EAGLE-794: Summary: Enable publish bolt parallelism Key: EAGLE-794 URL: https://issues.apache.org/jira/browse/EAGLE-794 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 Currently the publish is using shuffle grouping, we cannot enable parallelism for publish since we may have local cache which is unavailable across the storm cluster. We are going to use the same strategy as router bolts to alert bolts, which is using field grouping (define empty list of field) and dispatch tuple according to group by fields hashing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-758) Add tuple log for spout & alert bolt
[ https://issues.apache.org/jira/browse/EAGLE-758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-758. -- Resolution: Fixed > Add tuple log for spout & alert bolt > > > Key: EAGLE-758 > URL: https://issues.apache.org/jira/browse/EAGLE-758 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > Without tuple log for spout & alert bolt, it is hard to troubleshoot event > missing issue, we don't know whether the spout/alert process issue or no > event come. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-758) Add tuple log for spout & alert bolt
Garrett Li created EAGLE-758: Summary: Add tuple log for spout & alert bolt Key: EAGLE-758 URL: https://issues.apache.org/jira/browse/EAGLE-758 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 Without tuple log for spout & alert bolt, it is hard to troubleshoot event missing issue, we don't know whether the spout/alert process issue or no event come. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (EAGLE-741) Make publishment settings both policy & stream awareness
[ https://issues.apache.org/jira/browse/EAGLE-741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li updated EAGLE-741: - Summary: Make publishment settings both policy & stream awareness (was: Make publishment both policy & stream specific) > Make publishment settings both policy & stream awareness > > > Key: EAGLE-741 > URL: https://issues.apache.org/jira/browse/EAGLE-741 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > Currently our publishment is defined policy specific, we cannot publish the > alert into different places for one policy although there are multiple output > streams for this policy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-741) Make publishment both policy & stream specific
Garrett Li created EAGLE-741: Summary: Make publishment both policy & stream specific Key: EAGLE-741 URL: https://issues.apache.org/jira/browse/EAGLE-741 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 Currently our publishment is defined policy specific, we cannot publish the alert into different places for one policy although there are multiple output streams for this policy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-713) Publish & Router bolts spec change is not thread safe
Garrett Li created EAGLE-713: Summary: Publish & Router bolts spec change is not thread safe Key: EAGLE-713 URL: https://issues.apache.org/jira/browse/EAGLE-713 Project: Eagle Issue Type: Bug Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 Publish & Router bolts spec change is not thread safe, 2 concurrent spec change event will causes the policy has duplicated publishment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-640) Publishment reload may introduce NPE
[ https://issues.apache.org/jira/browse/EAGLE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-640. -- Resolution: Fixed > Publishment reload may introduce NPE > > > Key: EAGLE-640 > URL: https://issues.apache.org/jira/browse/EAGLE-640 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > > 2016-10-18 23:59:12.128 o.a.c.f.r.c.NodeCache [ERROR] Calling listener > java.lang.NullPointerException > at > org.apache.eagle.alert.engine.publisher.impl.AlertKafkaPublisher.update(AlertKafkaPublisher.java:78) > ~[stormjar.jar:?] > at > org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.onPublishChange(AlertPublisherImpl.java:164) > ~[stormjar.jar:?] > at > org.apache.eagle.alert.engine.runner.AlertPublisherBolt.onAlertPublishSpecChange(AlertPublisherBolt.java:109) > ~[stormjar.jar:?] > at > org.apache.eagle.alert.engine.coordinator.impl.AbstractMetadataChangeNotifyService.lambda$notifyAlertPublishBolt$3(AbstractMetadataChangeNotifyService.java:102) > ~[stormjar.jar:?] > at java.util.ArrayList.forEach(ArrayList.java:1249) > ~[?:1.8.0_92-internal] > at > org.apache.eagle.alert.engine.coordinator.impl.AbstractMetadataChangeNotifyService.notifyAlertPublishBolt(AbstractMetadataChangeNotifyService.java:102) > ~[stormjar.jar:?] > at > org.apache.eagle.alert.engine.coordinator.impl.ZKMetadataChangeNotifyService.onNewConfig(ZKMetadataChangeNotifyService.java:132) > ~[stormjar.jar:?] > at > org.apache.eagle.alert.config.ConfigBusConsumer.lambda$new$0(ConfigBusConsumer.java:44) > ~[stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:293) > [stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:287) > [stormjar.jar:?] > at > org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92) > [stormjar.jar:?] > at > com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299) > [stormjar.jar:?] > at > org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83) > [stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache.setNewData(NodeCache.java:284) > [stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache.processBackgroundResult(NodeCache.java:252) > [stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache.access$300(NodeCache.java:53) > [stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache$3.processResult(NodeCache.java:111) > [stormjar.jar:?] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:730) > [stormjar.jar:?] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:516) > [stormjar.jar:?] > at > org.apache.curator.framework.imps.GetDataBuilderImpl$3.processResult(GetDataBuilderImpl.java:254) > [stormjar.jar:?] > at > org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561) > [stormjar.jar:?] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) > [stormjar.jar:?] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-648) Stream router does not work properly sometimes
[ https://issues.apache.org/jira/browse/EAGLE-648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-648. -- Resolution: Fixed > Stream router does not work properly sometimes > -- > > Key: EAGLE-648 > URL: https://issues.apache.org/jira/browse/EAGLE-648 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > Stream router does not work properly when multiple policies use the same > input stream with the same partition definition. It is because stream router > bolt output maintains a map of the route specs, but the key of the map is > streamId + partition information. The monitor stream may override each other > if the streamId and partition are the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-660) Stream delete mongo implementation is not working
[ https://issues.apache.org/jira/browse/EAGLE-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-660. -- Resolution: Fixed > Stream delete mongo implementation is not working > - > > Key: EAGLE-660 > URL: https://issues.apache.org/jira/browse/EAGLE-660 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > we are using name rather than streamId to delete stream, it is incorrect -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-648) Stream router does not work properly sometimes
Garrett Li created EAGLE-648: Summary: Stream router does not work properly sometimes Key: EAGLE-648 URL: https://issues.apache.org/jira/browse/EAGLE-648 Project: Eagle Issue Type: Bug Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 Stream router does not work properly when multiple policies use the same input stream with the same partition definition. It is because stream router bolt output maintains a map of the route specs, but the key of the map is streamId + partition information. The monitor stream may override each other if the streamId and partition are the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-641) Publishment reload may introduce NPE
[ https://issues.apache.org/jira/browse/EAGLE-641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-641. -- Resolution: Fixed > Publishment reload may introduce NPE > > > Key: EAGLE-641 > URL: https://issues.apache.org/jira/browse/EAGLE-641 > Project: Eagle > Issue Type: Bug >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > > 2016-10-18 23:59:12.128 o.a.c.f.r.c.NodeCache [ERROR] Calling listener > java.lang.NullPointerException > at > org.apache.eagle.alert.engine.publisher.impl.AlertKafkaPublisher.update(AlertKafkaPublisher.java:78) > ~[stormjar.jar:?] > at > org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.onPublishChange(AlertPublisherImpl.java:164) > ~[stormjar.jar:?] > at > org.apache.eagle.alert.engine.runner.AlertPublisherBolt.onAlertPublishSpecChange(AlertPublisherBolt.java:109) > ~[stormjar.jar:?] > at > org.apache.eagle.alert.engine.coordinator.impl.AbstractMetadataChangeNotifyService.lambda$notifyAlertPublishBolt$3(AbstractMetadataChangeNotifyService.java:102) > ~[stormjar.jar:?] > at java.util.ArrayList.forEach(ArrayList.java:1249) > ~[?:1.8.0_92-internal] > at > org.apache.eagle.alert.engine.coordinator.impl.AbstractMetadataChangeNotifyService.notifyAlertPublishBolt(AbstractMetadataChangeNotifyService.java:102) > ~[stormjar.jar:?] > at > org.apache.eagle.alert.engine.coordinator.impl.ZKMetadataChangeNotifyService.onNewConfig(ZKMetadataChangeNotifyService.java:132) > ~[stormjar.jar:?] > at > org.apache.eagle.alert.config.ConfigBusConsumer.lambda$new$0(ConfigBusConsumer.java:44) > ~[stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:293) > [stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:287) > [stormjar.jar:?] > at > org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92) > [stormjar.jar:?] > at > com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299) > [stormjar.jar:?] > at > org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83) > [stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache.setNewData(NodeCache.java:284) > [stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache.processBackgroundResult(NodeCache.java:252) > [stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache.access$300(NodeCache.java:53) > [stormjar.jar:?] > at > org.apache.curator.framework.recipes.cache.NodeCache$3.processResult(NodeCache.java:111) > [stormjar.jar:?] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:730) > [stormjar.jar:?] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:516) > [stormjar.jar:?] > at > org.apache.curator.framework.imps.GetDataBuilderImpl$3.processResult(GetDataBuilderImpl.java:254) > [stormjar.jar:?] > at > org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561) > [stormjar.jar:?] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) > [stormjar.jar:?] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-641) Publishment reload may introduce NPE
Garrett Li created EAGLE-641: Summary: Publishment reload may introduce NPE Key: EAGLE-641 URL: https://issues.apache.org/jira/browse/EAGLE-641 Project: Eagle Issue Type: Bug Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li 2016-10-18 23:59:12.128 o.a.c.f.r.c.NodeCache [ERROR] Calling listener java.lang.NullPointerException at org.apache.eagle.alert.engine.publisher.impl.AlertKafkaPublisher.update(AlertKafkaPublisher.java:78) ~[stormjar.jar:?] at org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.onPublishChange(AlertPublisherImpl.java:164) ~[stormjar.jar:?] at org.apache.eagle.alert.engine.runner.AlertPublisherBolt.onAlertPublishSpecChange(AlertPublisherBolt.java:109) ~[stormjar.jar:?] at org.apache.eagle.alert.engine.coordinator.impl.AbstractMetadataChangeNotifyService.lambda$notifyAlertPublishBolt$3(AbstractMetadataChangeNotifyService.java:102) ~[stormjar.jar:?] at java.util.ArrayList.forEach(ArrayList.java:1249) ~[?:1.8.0_92-internal] at org.apache.eagle.alert.engine.coordinator.impl.AbstractMetadataChangeNotifyService.notifyAlertPublishBolt(AbstractMetadataChangeNotifyService.java:102) ~[stormjar.jar:?] at org.apache.eagle.alert.engine.coordinator.impl.ZKMetadataChangeNotifyService.onNewConfig(ZKMetadataChangeNotifyService.java:132) ~[stormjar.jar:?] at org.apache.eagle.alert.config.ConfigBusConsumer.lambda$new$0(ConfigBusConsumer.java:44) ~[stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:293) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:287) [stormjar.jar:?] at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92) [stormjar.jar:?] at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299) [stormjar.jar:?] at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache.setNewData(NodeCache.java:284) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache.processBackgroundResult(NodeCache.java:252) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache.access$300(NodeCache.java:53) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache$3.processResult(NodeCache.java:111) [stormjar.jar:?] at org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:730) [stormjar.jar:?] at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:516) [stormjar.jar:?] at org.apache.curator.framework.imps.GetDataBuilderImpl$3.processResult(GetDataBuilderImpl.java:254) [stormjar.jar:?] at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561) [stormjar.jar:?] at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) [stormjar.jar:?] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-594) Remove raw alert from kafka publisher
[ https://issues.apache.org/jira/browse/EAGLE-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-594. -- Resolution: Fixed > Remove raw alert from kafka publisher > - > > Key: EAGLE-594 > URL: https://issues.apache.org/jira/browse/EAGLE-594 > Project: Eagle > Issue Type: Improvement >Reporter: Garrett Li >Assignee: Garrett Li >Priority: Minor > > We are leveraging configured deduplicator to dedup the duplicated alerts > before publish to kafka, email, slack, etc. However, sometimes we may still > want to keep the raw alerts in kafka. Here we have defined > rawAlertNamespaceLabel and rawAlertNamespaceValue custom fields to emit the > raw alerts into kafka. > However, these configured namespace concept is ebay/sherlock specific, we > should remove it from eagle and use it ebay/sherlock extended kafka publisher. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-640) Publishment reload may introduce NPE
Garrett Li created EAGLE-640: Summary: Publishment reload may introduce NPE Key: EAGLE-640 URL: https://issues.apache.org/jira/browse/EAGLE-640 Project: Eagle Issue Type: Bug Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li 2016-10-18 23:59:12.128 o.a.c.f.r.c.NodeCache [ERROR] Calling listener java.lang.NullPointerException at org.apache.eagle.alert.engine.publisher.impl.AlertKafkaPublisher.update(AlertKafkaPublisher.java:78) ~[stormjar.jar:?] at org.apache.eagle.alert.engine.publisher.impl.AlertPublisherImpl.onPublishChange(AlertPublisherImpl.java:164) ~[stormjar.jar:?] at org.apache.eagle.alert.engine.runner.AlertPublisherBolt.onAlertPublishSpecChange(AlertPublisherBolt.java:109) ~[stormjar.jar:?] at org.apache.eagle.alert.engine.coordinator.impl.AbstractMetadataChangeNotifyService.lambda$notifyAlertPublishBolt$3(AbstractMetadataChangeNotifyService.java:102) ~[stormjar.jar:?] at java.util.ArrayList.forEach(ArrayList.java:1249) ~[?:1.8.0_92-internal] at org.apache.eagle.alert.engine.coordinator.impl.AbstractMetadataChangeNotifyService.notifyAlertPublishBolt(AbstractMetadataChangeNotifyService.java:102) ~[stormjar.jar:?] at org.apache.eagle.alert.engine.coordinator.impl.ZKMetadataChangeNotifyService.onNewConfig(ZKMetadataChangeNotifyService.java:132) ~[stormjar.jar:?] at org.apache.eagle.alert.config.ConfigBusConsumer.lambda$new$0(ConfigBusConsumer.java:44) ~[stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:293) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache$4.apply(NodeCache.java:287) [stormjar.jar:?] at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92) [stormjar.jar:?] at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299) [stormjar.jar:?] at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache.setNewData(NodeCache.java:284) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache.processBackgroundResult(NodeCache.java:252) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache.access$300(NodeCache.java:53) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.NodeCache$3.processResult(NodeCache.java:111) [stormjar.jar:?] at org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:730) [stormjar.jar:?] at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:516) [stormjar.jar:?] at org.apache.curator.framework.imps.GetDataBuilderImpl$3.processResult(GetDataBuilderImpl.java:254) [stormjar.jar:?] at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561) [stormjar.jar:?] at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) [stormjar.jar:?] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (EAGLE-531) Dedup alerts according to state change
[ https://issues.apache.org/jira/browse/EAGLE-531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li updated EAGLE-531: - Description: Currently the dedup is only for specified fields dedup, it cannot support dedup the alert according to dedup fields along with one specific field value changes (such as state change). This change is support both cases. If only specified fields for dedup, we will according to specified fields and duration time to filter duplicated alerts. If specified state field along with dedup fields, we will check state filed value change to filter duplicated alerts. was:This is for stateful and stateless logs > Dedup alerts according to state change > -- > > Key: EAGLE-531 > URL: https://issues.apache.org/jira/browse/EAGLE-531 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > Currently the dedup is only for specified fields dedup, it cannot support > dedup the alert according to dedup fields along with one specific field value > changes (such as state change). This change is support both cases. > If only specified fields for dedup, we will according to specified fields and > duration time to filter duplicated alerts. > If specified state field along with dedup fields, we will check state filed > value change to filter duplicated alerts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (EAGLE-594) Remove raw alert from kafka publisher
[ https://issues.apache.org/jira/browse/EAGLE-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li updated EAGLE-594: - Summary: Remove raw alert from kafka publisher (was: Skip raw alert in kafka publisher if not configured) > Remove raw alert from kafka publisher > - > > Key: EAGLE-594 > URL: https://issues.apache.org/jira/browse/EAGLE-594 > Project: Eagle > Issue Type: Improvement >Reporter: Garrett Li >Assignee: Garrett Li >Priority: Minor > > We are leveraging configured deduplicator to dedup the duplicated alerts > before publish to kafka, email, slack, etc. However, sometimes we may still > want to keep the raw alerts in kafka. Here we have defined > rawAlertNamespaceLabel and rawAlertNamespaceValue custom fields to emit the > raw alerts into kafka. However, we still want to skip it if these 2 fields > are not configured. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-594) Skip raw alert in kafka publisher if not configured
Garrett Li created EAGLE-594: Summary: Skip raw alert in kafka publisher if not configured Key: EAGLE-594 URL: https://issues.apache.org/jira/browse/EAGLE-594 Project: Eagle Issue Type: Improvement Reporter: Garrett Li Assignee: Garrett Li Priority: Minor We are leveraging configured deduplicator to dedup the duplicated alerts before publish to kafka, email, slack, etc. However, sometimes we may still want to keep the raw alerts in kafka. Here we have defined rawAlertNamespaceLabel and rawAlertNamespaceValue custom fields to emit the raw alerts into kafka. However, we still want to skip it if these 2 fields are not configured. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-579) Cloned stream alert event missed stream name
[ https://issues.apache.org/jira/browse/EAGLE-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-579. -- Resolution: Fixed > Cloned stream alert event missed stream name > > > Key: EAGLE-579 > URL: https://issues.apache.org/jira/browse/EAGLE-579 > Project: Eagle > Issue Type: Bug >Reporter: Garrett Li >Assignee: Garrett Li > > We are trying to clone stream alert event if there is a state transition (we > need to update last state dedup count and first occurrence time). If the > cloned stream alert event missed the stream name, the publishment cannot work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-578) Make dedup state value case insensitive
[ https://issues.apache.org/jira/browse/EAGLE-578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-578. -- Resolution: Fixed > Make dedup state value case insensitive > --- > > Key: EAGLE-578 > URL: https://issues.apache.org/jira/browse/EAGLE-578 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > Currently our dedup feature supports both stateful and stateless dedup. For > stateful dedup, we are relying the state transition to do the dedup. However, > the state might be closed or CLOSED, we'd better to make it case insensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-579) Cloned stream alert event missed stream name
Garrett Li created EAGLE-579: Summary: Cloned stream alert event missed stream name Key: EAGLE-579 URL: https://issues.apache.org/jira/browse/EAGLE-579 Project: Eagle Issue Type: Bug Reporter: Garrett Li We are trying to clone stream alert event if there is a state transition (we need to update last state dedup count and first occurrence time). If the cloned stream alert event missed the stream name, the publishment cannot work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-578) Make dedup state value case insensitive
Garrett Li created EAGLE-578: Summary: Make dedup state value case insensitive Key: EAGLE-578 URL: https://issues.apache.org/jira/browse/EAGLE-578 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 Currently our dedup feature supports both stateful and stateless dedup. For stateful dedup, we are relying the state transition to do the dedup. However, the state might be closed or CLOSED, we'd better to make it case insensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-576) Dedup enhancements
[ https://issues.apache.org/jira/browse/EAGLE-576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-576. -- Resolution: Fixed merged > Dedup enhancements > -- > > Key: EAGLE-576 > URL: https://issues.apache.org/jira/browse/EAGLE-576 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > 1. per publishment per cache, without global cache > 2. support dedup without state field > 3. emit raw alert into configured namespace -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (EAGLE-576) Dedup enhancements
[ https://issues.apache.org/jira/browse/EAGLE-576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li updated EAGLE-576: - Description: 1. per publishment per cache, without global cache 2. support dedup without state field 3. emit raw alert into configured namespace was: 1. per publishment per cache, without global cache 2. support dedup without state field > Dedup enhancements > -- > > Key: EAGLE-576 > URL: https://issues.apache.org/jira/browse/EAGLE-576 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > 1. per publishment per cache, without global cache > 2. support dedup without state field > 3. emit raw alert into configured namespace -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-576) Dedup enhancements
Garrett Li created EAGLE-576: Summary: Dedup enhancements Key: EAGLE-576 URL: https://issues.apache.org/jira/browse/EAGLE-576 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 1. per publishment per cache, without global cache 2. support dedup without state field -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-566) add dedup fields into publishment compare consideration
Garrett Li created EAGLE-566: Summary: add dedup fields into publishment compare consideration Key: EAGLE-566 URL: https://issues.apache.org/jira/browse/EAGLE-566 Project: Eagle Issue Type: Bug Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 add dedup fields into publishment compare consideration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-519) No data / gap alert enhancement
[ https://issues.apache.org/jira/browse/EAGLE-519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-519. -- Resolution: Fixed > No data / gap alert enhancement > --- > > Key: EAGLE-519 > URL: https://issues.apache.org/jira/browse/EAGLE-519 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > 1. no data for an entire stream won't trigger gap alert (use local time & > batch window) > 2. no data policies definition is too complex, make the nodata offer optional > settings in stream definition > 3. bolt parallellism will cause multiple alerts in the same time window (use > loopback) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-531) Dedup alerts according to state change
[ https://issues.apache.org/jira/browse/EAGLE-531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-531. -- Resolution: Fixed > Dedup alerts according to state change > -- > > Key: EAGLE-531 > URL: https://issues.apache.org/jira/browse/EAGLE-531 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > https://wiki.vip.corp.ebay.com/pages/viewpage.action?pageId=361488851 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (EAGLE-531) Dedup alerts according to state change
[ https://issues.apache.org/jira/browse/EAGLE-531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li updated EAGLE-531: - Description: https://wiki.vip.corp.ebay.com/pages/viewpage.action?pageId=361488851 > Dedup alerts according to state change > -- > > Key: EAGLE-531 > URL: https://issues.apache.org/jira/browse/EAGLE-531 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > > https://wiki.vip.corp.ebay.com/pages/viewpage.action?pageId=361488851 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (EAGLE-544) Enhance dedup to support extended deduplicator
[ https://issues.apache.org/jira/browse/EAGLE-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li closed EAGLE-544. > Enhance dedup to support extended deduplicator > -- > > Key: EAGLE-544 > URL: https://issues.apache.org/jira/browse/EAGLE-544 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-544) Enhance dedup to support extended deduplicator
[ https://issues.apache.org/jira/browse/EAGLE-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-544. -- Resolution: Fixed created pr https://github.com/apache/incubator-eagle/pull/437 > Enhance dedup to support extended deduplicator > -- > > Key: EAGLE-544 > URL: https://issues.apache.org/jira/browse/EAGLE-544 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Fix For: v0.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-544) Enhance dedup to support extended deduplicator
Garrett Li created EAGLE-544: Summary: Enhance dedup to support extended deduplicator Key: EAGLE-544 URL: https://issues.apache.org/jira/browse/EAGLE-544 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-531) Dedup alerts according to state change
Garrett Li created EAGLE-531: Summary: Dedup alerts according to state change Key: EAGLE-531 URL: https://issues.apache.org/jira/browse/EAGLE-531 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-519) No data / gap alert enhancement
Garrett Li created EAGLE-519: Summary: No data / gap alert enhancement Key: EAGLE-519 URL: https://issues.apache.org/jira/browse/EAGLE-519 Project: Eagle Issue Type: Improvement Affects Versions: v0.5.0 Reporter: Garrett Li Assignee: Garrett Li Fix For: v0.5.0 1. no data for an entire stream won't trigger gap alert (use local time & batch window) 2. no data policies definition is too complex, make the nodata offer optional settings in stream definition 3. bolt parallellism will cause multiple alerts in the same time window (use loopback) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-435) Coordiantor schedule operation must be exclusive in distributed deployment
[ https://issues.apache.org/jira/browse/EAGLE-435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-435. -- Resolution: Fixed merged to upstream > Coordiantor schedule operation must be exclusive in distributed deployment > -- > > Key: EAGLE-435 > URL: https://issues.apache.org/jira/browse/EAGLE-435 > Project: Eagle > Issue Type: Improvement >Affects Versions: v0.5.0 >Reporter: Garrett Li >Assignee: Garrett Li > Labels: alert_engine > Fix For: v0.5.0 > > Original Estimate: 24h > Remaining Estimate: 24h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (EAGLE-434) BUG: periodically trigger schedule in service mode
[ https://issues.apache.org/jira/browse/EAGLE-434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li resolved EAGLE-434. -- Resolution: Fixed merged the code to upstream > BUG: periodically trigger schedule in service mode > -- > > Key: EAGLE-434 > URL: https://issues.apache.org/jira/browse/EAGLE-434 > Project: Eagle > Issue Type: Bug >Reporter: Garrett Li >Assignee: Garrett Li > Labels: alert_engine > Original Estimate: 24h > Remaining Estimate: 24h > > the periodical trigger schedule is not started -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-439) Introduce policy enable/disable support
Garrett Li created EAGLE-439: Summary: Introduce policy enable/disable support Key: EAGLE-439 URL: https://issues.apache.org/jira/browse/EAGLE-439 Project: Eagle Issue Type: Improvement Reporter: Garrett Li Assignee: Garrett Li -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (EAGLE-434) BUG: periodically trigger schedule in service mode
[ https://issues.apache.org/jira/browse/EAGLE-434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li reassigned EAGLE-434: Assignee: Garrett Li > BUG: periodically trigger schedule in service mode > -- > > Key: EAGLE-434 > URL: https://issues.apache.org/jira/browse/EAGLE-434 > Project: Eagle > Issue Type: Bug >Reporter: Garrett Li >Assignee: Garrett Li > Labels: alert_engine > Original Estimate: 24h > Remaining Estimate: 24h > > the periodical trigger schedule is not started -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (EAGLE-435) Coordiantor schedule operation must be exclusive in distributed deployment
[ https://issues.apache.org/jira/browse/EAGLE-435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li updated EAGLE-435: - Issue Type: Improvement (was: Bug) Summary: Coordiantor schedule operation must be exclusive in distributed deployment (was: Enhancement: coordiantor schedule operation must be exclusive in distributed deployment) > Coordiantor schedule operation must be exclusive in distributed deployment > -- > > Key: EAGLE-435 > URL: https://issues.apache.org/jira/browse/EAGLE-435 > Project: Eagle > Issue Type: Improvement >Reporter: Garrett Li > Labels: alert_engine > Original Estimate: 24h > Remaining Estimate: 24h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (EAGLE-435) Enhancement: coordiantor schedule operation must be exclusive in distributed deployment
Garrett Li created EAGLE-435: Summary: Enhancement: coordiantor schedule operation must be exclusive in distributed deployment Key: EAGLE-435 URL: https://issues.apache.org/jira/browse/EAGLE-435 Project: Eagle Issue Type: Bug Reporter: Garrett Li -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (EAGLE-434) BUG: periodically trigger schedule in service mode
[ https://issues.apache.org/jira/browse/EAGLE-434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Li updated EAGLE-434: - Labels: alert_engine (was: ) > BUG: periodically trigger schedule in service mode > -- > > Key: EAGLE-434 > URL: https://issues.apache.org/jira/browse/EAGLE-434 > Project: Eagle > Issue Type: Bug >Reporter: Garrett Li > Labels: alert_engine > Original Estimate: 24h > Remaining Estimate: 24h > > the periodical trigger schedule is not started -- This message was sent by Atlassian JIRA (v6.3.4#6332)
