[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-29 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/4172 Hi @wuchong . I am not an expert in Janino and how it works but I do not think you need Janino's classloader at any point. Using the `open()` of the `CEPOperator` you just need to compile the code of th

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-29 Thread wuchong
Github user wuchong commented on the issue: https://github.com/apache/flink/pull/4172 Yes @twalthr , you are right, the cause is different classloader. The compiled class is under Janino's custom ClassLoader. Using the current thread's classloader or user code classloader can't load t

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-28 Thread twalthr
Github user twalthr commented on the issue: https://github.com/apache/flink/pull/4172 @wuchong this sounds like a classloader issues. Where does `SerializationUtils` come from? Maybe it is because of the wrong class loader. In `CRowProcessRunner` we are using the user code classloader

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-28 Thread wuchong
Github user wuchong commented on the issue: https://github.com/apache/flink/pull/4172 @kl0u I mean the compiled class (i.e. the class of the code generated `IterativeCondition`) can't be found while deserializing. Maybe it's a problem of Janino compiler. cc @fhueske do you know why ?

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-28 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/4172 @wuchong Yes this is what I had in mind. Could you tell me which class is not found? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-28 Thread wuchong
Github user wuchong commented on the issue: https://github.com/apache/flink/pull/4172 @kl0u , do you mean compile code to `IterativeCondition` and set it to `StateTransition.newCondition` field instead of the original wrapper? And it will be serialized during snapshot and deserialize

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-28 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/4172 @dianfu So this is a plan that can work to avoid having to generate the code of the `IterativeCondition` every time. This came also after discussion with @fhueske who also explained me how things are do

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-27 Thread wuchong
Github user wuchong commented on the issue: https://github.com/apache/flink/pull/4172 @kl0u I'm fine with this. I want to make sure that in order to support FLINK-6938, we have to call `open()` on every condition after every time we call `getNFA()`, right? --- If your project is

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-27 Thread dianfu
Github user dianfu commented on the issue: https://github.com/apache/flink/pull/4172 @kl0u Thanks for the comments. I think that makes sense. We can revisit this optimization after adding `Pattern` at runtime is supported :) --- If your project is set up for it, you can reply to this

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-27 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/4172 @dianfu and I also include @wuchong on this as these two are related. The way I see it is that by not serializing the condition and the states, you are trying to gain some speed, especially whe

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-27 Thread dianfu
Github user dianfu commented on the issue: https://github.com/apache/flink/pull/4172 @kl0u Thanks for your suggestions. I think the optimizations in this PR are straight forward. IMO, the changes which may have something to do with other features ongoing are all in `AbstractKeyedCEPP

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-27 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/4172 Hi @dianfu . Thanks for updating the PR. I left some comments on the related JIRA (https://issues.apache.org/jira/browse/FLINK-6983) and the related https://issues.apache.org/jira/browse/FLINK-

[GitHub] flink issue #4172: [FLINK-6983] [cep] Do not serialize States with NFA

2017-06-26 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/4172 Hi @dianfu , this branch seems to be broken. Most of the migration tests fail when you run them locally. I will keep on checking out the branch, just to see the general idea of the change because this m