Re: Confusion about multiple use of one ValueState

2016-05-13 Thread nsengupta
Sorry, Balaji! Somehow, I missed this particular post of yours. Please ignore my last mail, where I am asking the same question. -- Nirmalya -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Confusion-about-multiple-use-of-one-ValueState

Re: Confusion about multiple use of one ValueState

2016-05-13 Thread Nirmalya Sengupta
Hello Balaji Yes. The State holder 'sum' in my example is actually created outside the Mapper objects; so it stays where it is. I am creating 'var's inside the Mapper objects to _refer_ to the same object, irrespective of multiplicity of the Mappers. The _open_ function is helping to make that a

Re: Confusion about multiple use of one ValueState

2016-05-13 Thread Balaji Rajagopalan
I wrote a simple helper class, the redis connection are initialized in the constructor and there are set and get methods to store and retreive values from your map functions. If you find any better way to do this please share :). I am using redis scala client. object class RedisHelper { val re

Re: Confusion about multiple use of one ValueState

2016-05-13 Thread Nirmalya Sengupta
Hello Balaji , Thanks for your reply. This confirms my earlier assumption that one of usual ways to do it is to hold and nurture the application-state in an external body; in your case: Redis. So, I am trying to understand how does one share the handle to this external body amongst partitions: do

Re: Confusion about multiple use of one ValueState

2016-05-13 Thread Balaji Rajagopalan
Even thought there are multiple instance of map object transient value object state is accessible across the object, so as the stream is flowing in the value can be updated based on application logic. On Fri, May 13, 2016 at 11:26 AM, Balaji Rajagopalan < balaji.rajagopa...@olacabs.com> wrote: >

Re: Confusion about multiple use of one ValueState

2016-05-12 Thread Balaji Rajagopalan
I don't think the valuestate defined in one map function is accessible in other map function this is my understanding, also you need to be aware there will be instance of map function created for each of your tuple in your stream, I had a similar use case where I had to pass in some state from one

Confusion about multiple use of one ValueState

2016-05-12 Thread Nirmalya Sengupta
Hello all, Let's say I want to hold some state value derived during one transformation, and then use that same state value in a subsequent transformation? For example: myStream .keyBy(fieldID) // Some field ID, may be 0 .map(new MyStatefulMapper()) .map(new MySubsequentMapper()) Now, I defi