[
https://issues.apache.org/jira/browse/EAGLE-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085095#comment-15085095
]
ASF GitHub Bot commented on EAGLE-39:
-------------------------------------
GitHub user yonzhang opened a pull request:
https://github.com/apache/incubator-eagle/pull/55
EAGLE-39 Policy State Management
executor state management mainly for policy state durability for failover
1. basic implementation of state snapshot, delta event persistence,
snapshot recover, delta event recovery
2. by default it is disabled
3. need improve: batch insert delta events to kafka, and batch ack storm
4. need performance test further.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yonzhang/incubator-eagle policystate_squash
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-eagle/pull/55.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #55
----
commit 95cd0eb381e7ecf219deb32b9baf7da009dcdcc2
Author: yonzhang <yonzh...@ebay.com>
Date: 2015-12-22T22:57:47Z
test siddhi aggregation and snapshot and restore
commit 1df4a7fd7a123bdbbdae49bbbe296add9b67be21
Author: yonzhang <yonzh...@ebay.com>
Date: 2016-01-06T06:36:35Z
executor state management
commit 94e92e3b89f078311f3f2c9c9328c077ea91d5b4
Author: yonzhang <yonzh...@ebay.com>
Date: 2016-01-06T06:36:35Z
executor state management
commit 13f80d60fdc8890eed5fc2cd6438f528c03cebe2
Author: yonzhang <yonzh...@ebay.com>
Date: 2016-01-06T06:58:57Z
merged
----
> Policy State Management for robustness - replay events upon node crash
> ----------------------------------------------------------------------
>
> Key: EAGLE-39
> URL: https://issues.apache.org/jira/browse/EAGLE-39
> Project: Eagle
> Issue Type: New Feature
> Affects Versions: 0.3.0
> Reporter: Edward Zhang
> Assignee: Edward Zhang
> Fix For: 0.3.0
>
>
> Most useful policies are stateful, for example slide-window based policy is
> evaluated not only for current event but also for a sequential set of events
> happening before current event.
> When the node which runs the stateful policy crashes, the policy would be
> built up in another node by Eagle, but problem is we never see the events
> which were populated for this policy before node crash.
> The problem sometimes will cause serious issues as it may miss some alerts.
> Think about a policy which takes one month's data is gone because of the node
> crash, how do we recover the one-month data for continuous evaluation.
> Unfortunately neither Storm as stream infrastructure nor Siddhi as CEP engine
> provides state management of intermediate data generated in the program.
> Storm can bring up another bolt when one bolt goes down, but it is not
> responsible for replay the events which application were previously received.
> Siddhi does not provide any state recover when CEP runtime goes down.
> Eagle should provide some API for easy state management between monitoring
> application and CEP engine. That would make a lot of sense when we define
> complex policies.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
