Re: Standalone HA cluster: Fatal error occurred in the cluster entrypoint.

2018-11-16 Thread Olga Luganska
luster again all the the checkpoint and job graph data will be erased and basacly you will start a new cluster... It's happened to me allot on a 1.5.x On a 1.6 things are running perfect . I'm not sure way this error is back again on 1.6.1 ? On Fri, 16 Nov 2018, 0:42 Olga Luganska mailto:trebl...@ho

Standalone HA cluster: Fatal error occurred in the cluster entrypoint.

2018-11-15 Thread Olga Luganska
Hello, I am running flink 1.6.1 standalone HA cluster. Today I am unable to start cluster because of "Fatal error in cluster entrypoint" (I used to see this error when running flink 1.5 version, after upgrade to 1.6.1 (which had a fix for this bug) everything worked well for a while) Question:

Enabling Flink’s checkpointing

2018-11-15 Thread Olga Luganska
Hello, By reading Flink documentation I see that to enable checkpointing we need to: 1. Enable checkpointing at the execution environment. 2. Make sure that your source/sink implements either CheckpointedFunction or ListCheckpointed interfaces? Is #2 a must, and how checkpointing mechanism is

Re: FlinkKafkaProducer and Confluent Schema Registry

2018-11-08 Thread Olga Luganska
://github.com/apache/flink/pull/6259 On Mon, 22 Oct 2018 at 16:51, Kostas Kloudas mailto:k.klou...@data-artisans.com>> wrote: Hi Olga, Sorry for the late reply. I think that Gordon (cc’ed) could be able to answer your question. Cheers, Kostas On Oct 13, 2018, at 3:10 PM, Olga Luganska mailto

Re: FlinkKafkaProducer and Confluent Schema Registry

2018-10-13 Thread Olga Luganska
Any suggestions? Thank you Sent from my iPhone On Oct 9, 2018, at 9:28 PM, Olga Luganska mailto:trebl...@hotmail.com>> wrote: Hello, I would like to use Confluent Schema Registry in my streaming job. I was able to make it work with the help of generic Kafka producer and FlinkKafkaCo

FlinkKafkaProducer and Confluent Schema Registry

2018-10-09 Thread Olga Luganska
Hello, I would like to use Confluent Schema Registry in my streaming job. I was able to make it work with the help of generic Kafka producer and FlinkKafkaConsumer which is using ConfluentRegistryAvroDeserializationSchema. FlinkKafkaConsumer011 consumer = new FlinkKafkaConsumer011<>(MY_TOPIC,

Using several Kerberos keytabs in standalone cluster

2018-10-07 Thread Olga Luganska
Hello, According to the documentation, the security setup is shared by all the jobs on the same cluster, and if users need to use a different keytab, it is easily achievable in Yarn cluster setup by starting a new cluster with a different flink-conf.yaml Is it possible to setup a standalone

Flink support for multiple data centers

2018-10-01 Thread Olga Luganska
Hello, Does Flink support multiple data center implementation and failover procedures in case one of the data centers goes down? Another question I have is about data encryption. If application state which needs to be checkpointed contains data elements which are considered to be a