Just wondering is this on the client side in the flink Job? I rebooted the task and the job deployed correctly on another node.
Is there a specific ulimit that we should set for flink tasks nodes? org.apache.kafka.common.KafkaException: Failed to construct kafka consumer at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:799) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:650) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:630) at org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:58) at org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94) at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:504) at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.kafka.common.KafkaException: java.io.IOException: Too many open files at org.apache.kafka.common.network.Selector.<init>(Selector.java:154) at org.apache.kafka.common.network.Selector.<init>(Selector.java:188) at org.apache.kafka.common.network.Selector.<init>(Selector.java:192) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:722) ... 11 more Caused by: java.io.IOException: Too many open files at sun.nio.ch.IOUtil.makePipe(Native Method) at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:65) at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36) at java.nio.channels.Selector.open(Selector.java:227) at org.apache.kafka.common.network.Selector.<init>(Selector.java:152) ... 14 more
