[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-08-07 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/2218 Thanks for the reviews, squashed and merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-08-07 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/2218 @arunmahadevan +1 Nice work. Could you squash the commits into one? You can merge it by yourself, or please mention me if you want me to merge. --- If your project is set up for it, you

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-08-03 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/2218 @srdo , thanks for taking time to do an in-depth review. Have addressed your latest comments. --- If your project is set up for it, you can reply to this email and have your reply appear on Gi

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-08-01 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/2218 @srdo , @satishd, thanks for the review. Pushed some changes to address the latest review comments. --- If your project is set up for it, you can reply to this email and have your reply appea

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-07-27 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/2218 @srdo, addressed your comments, let me know if I missed something. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-07-24 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/2218 @ben-manes , thanks for the insights. Right now the custom logic is kind of straightforward. May be I will attempt to provide an alternate implementation on top of Caffeine which could be usefu

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-07-24 Thread ben-manes
Github user ben-manes commented on the issue: https://github.com/apache/storm/pull/2218 fyi, you can emulate pinning by setting the entry's weight to zero. Then the entry will not be evicted, but is eligible for expiration, etc. Of course zero weight entries means the cache allows mor

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-07-24 Thread satishd
Github user satishd commented on the issue: https://github.com/apache/storm/pull/2218 @HeartSaVioR I will go through this PR in next 1-2 days and put my comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-07-24 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/2218 After some more tests (with Caffeine for caching) I found some instances where the current partition (where entries are added or being removed) could get evicted causing issues in serialization

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-07-19 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2218 @arunmahadevan ... could you put that info on the design in the JIRA itself. easier for future reference. --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-07-18 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/2218 This builds on top of the existing state checkpointing mechanism (documented here - https://github.com/apache/storm/blob/master/docs/State-checkpointing.md). Theres nothing extra added to the

[GitHub] storm issue #2218: STORM-2614: Enhance stateful windowing to persist the win...

2017-07-17 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2218 Sounds like a checkpoint mechanism limited to windowed bolts. Could you provide a design document for this ? --- If your project is set up for it, you can reply to this email and have your repl