Using guava ImmutableMap Serializer from magro/kyro-serializers

2019-04-16 Thread vijikarthi
Hello, I am using Flink 1.7.2. I wrote a small application which uses KeyedProcessFunction to maintain application state. The state value object is using Guava's (version 18.0.50) ImmutableMap to create a copy of the map instance as part of its constructor. I am using "de.javakaffee:kryo-seriali

Re: Using guava ImmutableMap Serializer from magro/kyro-serializers

2019-04-17 Thread vijikarthi
Thanks JingsongLee. You are right. I have registered all the implementation class and it worked. -- Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

Checkpoint Vs Job Completion

2019-05-01 Thread vijikarthi
Hi, I have a question regarding how checkpoint works when a job is getting finished. Let's imagine that I have a custom sink implementation which is making use of checkpointing mechanism to commit the writes (i.e., data will be flushed only when commit API call is invoked and the invocation of c

Re: Checkpoint Vs Job Completion

2019-05-01 Thread vijikarthi
It looks like the problem statement is similar to what we have in "FLIP-34: Terminate/Suspend Job with Savepoint"? -- Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

Checkpoint / Two Phase Commit

2019-05-21 Thread vijikarthi
Question regarding end-to-end exactly once guarantee implementation using 2PC? As I understand how it operates, the pre-phase state is when the checkpoint is initiated and the checkpoint barrier advances from source to sink. Once the pre-phase is complete (and successful), then the next step in t

Re: Checkpoint / Two Phase Commit

2019-05-24 Thread vijikarthi
Pinging again on this thread to see if anyone has any recommendations? Particularly, I am interested to understand whether this scenario is also applicable for Kafka Connector? -- Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

Re: Yarn Integration Test

2016-06-19 Thread vijikarthi
Looks like I am running in to some sort of environment related issue (??) but the problem is driving me crazy. I was able to run the same test succesfully last week but now I am getting some wierd compile issue with the dependent module. Here is the complete stack trace. Amy help is much appreciate

Re: Shading Plugin + Hadoop Dependency

2016-06-24 Thread vijikarthi
Any modifications to "flink-shaded-include-yarn-tests" with respect to the dependency inclusion/exclusion does not get reflected in the "flink-yarn-tests" test case since the referenced packages from "flink-shaded-include-yarn-tests" are not being used (though the pom file establish the dependency)

Re: Flink using Yarn on MapR

2016-11-11 Thread vijikarthi
Flink security context gets initialized during the application start phase. As part of the initialization, the UserGroupInformation (UGI) instance is bootstrapped using the Hadoop configuration files (read: HADOOP_CONF_DIR or YARN_CONF_DIR environment variable is set). If the hadoop configuration (