James B. Fitzgerald created FLINK-14523: -------------------------------------------
Summary: Flink Stops Consuming from Kafka after Leader Election Key: FLINK-14523 URL: https://issues.apache.org/jira/browse/FLINK-14523 Project: Flink Issue Type: Bug Components: Connectors / Kafka Environment: In AWS we run the app on EMR with the following versions: * EMR Release - emr-5.15.0 * Hadoop distribution - Amazon 2.8.3 * Flink - Flink 1.4.2 We submit the job to the cluster as an EMR step using ***command-runner.jar*. We submit the job with the following arguments: {code:java} "Args": [ "flink", "run", "-m", "yarn-cluster", "-c", "com.salesforce.sde.streamingsearches.StreamingSearchesJob", "-yst", "-ys", "4", "-yn", "10", "-yjm", "2800", "-ytm", "2800", "-ynm", "streaming-searches-prod", "-d", "/home/hadoop/streaming-searches-1.0-SNAPSHOT.jar" ]{code} Additionally we build our application jar with Flink 1.4.2 and Kafka 0.11. Reporter: James B. Fitzgerald We have a Flink application running in AWS on EMR that streams input from a single Kafka topic. Whenever there is a Kafka leader election for any partition of the input topic, our Flink application stops consuming from Kafka entirely. To begin consuming from Kafka again the YARN app must be killed and restarted. We run this same application on premises and in AWS on EMR. We have only observed this behavior when it is running on EMR. -- This message was sent by Atlassian Jira (v8.3.4#803005)