Glad to hear that outcome. And no worries about the false alarm. Thank you for doing thorough testing, this is very helpful!
On Wed, Apr 28, 2021 at 1:04 PM Caizhi Weng <tsreape...@gmail.com> wrote: > After the investigation we found that this issue is caused by the > implementation of connector, not by the Flink framework. > > Sorry for the false alarm. > > Stephan Ewen <se...@apache.org> 于2021年4月28日周三 下午3:23写道： > > > @Caizhi and @Becket - let me reach out to you to jointly debug this > issue. > > > > I am wondering if there is some incorrect reporting of failed events? > > > > On Wed, Apr 28, 2021 at 8:53 AM Caizhi Weng <tsreape...@gmail.com> > wrote: > > > > > -1 > > > > > > We're testing this version on batch jobs with large (600~1000) > > parallelisms > > > and the following exception messages appear with high frequency: > > > > > > 2021-04-27 21:27:26 > > > org.apache.flink.util.FlinkException: An OperatorEvent from an > > > OperatorCoordinator to a task was lost. Triggering task failover to > > ensure > > > consistency. Event: '[NoMoreSplitEvent]', targetTask: <task name> - > > > execution #0 > > > at > > > > > > > > > org.apache.flink.runtime.operators.coordination.SubtaskGatewayImpl.lambda$sendEvent$0(SubtaskGatewayImpl.java:81) > > > at > > > > > > > > > java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) > > > at > > > > > > > > > java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) > > > at > > > > > > > > > java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442) > > > at > > > > > > > > > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:440) > > > at > > > > > > > > > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:208) > > > at > > > > > > > > > org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) > > > at > > > > > > > > > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) > > > at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) > > > at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) > > > at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) > > > at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) > > > at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) > > > at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) > > > at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) > > > at akka.actor.Actor$class.aroundReceive(Actor.scala:517) > > > at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) > > > at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) > > > at akka.actor.ActorCell.invoke(ActorCell.scala:561) > > > at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) > > > at akka.dispatch.Mailbox.run(Mailbox.scala:225) > > > at akka.dispatch.Mailbox.exec(Mailbox.scala:235) > > > at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > > > at > > > > > > > > > akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > > > at > akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > > > at > > > > > > > > > akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > > > > > > Becket Qin is investigating this issue. > > > > > >
