zxylmqx opened a new issue, #4838: URL: https://github.com/apache/seatunnel/issues/4838
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened 使用zeta引擎+mysqlcdc，同步mysql数据到doris时，提交一个任务可以正常运行，提交第二个任务会出现两个任务都不工作的情况，查看job详情，状态依然是running，但是已经不同步数据了 ### SeaTunnel Version 2.3.1 ### SeaTunnel Config ```conf seatunnel: engine: backup-count: 2 queue-type: blockingqueue print-execution-info-interval: 10 print-job-metrics-info-interval: 10 slot-service: dynamic-slot: true checkpoint: interval: 30000 timeout: 20000 max-concurrent: 5 tolerable-failure: 2 storage: type: hdfs max-retained: 3 plugin-config: namespace: /tmp/seatunnel/checkpoint_snapshot storage.type: hdfs fs.defaultFS: hdfs://bigdata:8022 # Ensure that the directory has written permission ``` ### Running Command ```shell ${SEATUNNEL_HOME}/bin/seatunnel.sh --config /tmp/dolphinscheduler/exec/process/yinghuo-common/9648406837856/9650140382816_23/2190/5162/seatunnel_2190_5162.conf --deploy-mode cluster -n debezium_source ``` ### Error Exception ```log 2023-05-26 16:23:00,831 INFO com.hazelcast.spi.impl.operationservice.impl.InvocationMonitor - [bigdata07]:5801 [seatunnel_mycluster] [5.1] Invocations:3 timeouts:1 backup-timeouts:0 2023-05-26 16:23:00,832 WARN org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job debezium_source (714388703002230790), Pipeline: [(1/1)] checkpoint have error, cancel the pipeline 2023-05-26 16:23:00,833 WARN com.hazelcast.internal.server.tcp.TcpServerConnection - [bigdata07]:5801 [seatunnel_mycluster] [5.1] Connection[id=65, /[10.6.1.219:5801](http://10.6.1.219:5801/)->/[10.6.1.16:39014](http://10.6.1.16:39014/), qualifier=null, endpoint=[bigdata05]:5801, remoteUuid=02a08838-4c85-4e91-882b-881f537228d2, alive=false, connectionType=MEMBER, planeIndex=0] closed. Reason: Exception in Connection[id=65, /[10.6.1.219:5801](http://10.6.1.219:5801/)->/[10.6.1.16:39014](http://10.6.1.16:39014/), qualifier=null, endpoint=[bigdata05]:5801, remoteUuid=02a08838-4c85-4e91-882b-881f537228d2, alive=true, connectionType=MEMBER, planeIndex=0], thread=checkpoint-coordinator-1/714388703002230790 java.nio.channels.ClosedByInterruptException: null at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) ~[?:1.8.0_352] at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:477) ~[?:1.8.0_352] at com.hazelcast.internal.networking.nio.NioOutboundPipeline.flushToSocket(NioOutboundPipeline.java:439) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.internal.networking.nio.NioOutboundPipeline.process(NioOutboundPipeline.java:324) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.internal.networking.nio.NioOutboundPipeline.executePipeline(NioOutboundPipeline.java:240) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.internal.networking.nio.NioOutboundPipeline.write(NioOutboundPipeline.java:218) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.internal.networking.nio.NioChannel.write(NioChannel.java:79) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.internal.server.tcp.TcpServerConnection.write(TcpServerConnection.java:222) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.spi.impl.operationservice.impl.OutboundOperationHandler.send(OutboundOperationHandler.java:59) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeRemote(Invocation.java:612) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:582) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:541) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:241) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:61) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.map.impl.proxy.MapProxySupport.invokeOperation(MapProxySupport.java:478) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.map.impl.proxy.MapProxySupport.getInternal(MapProxySupport.java:371) ~[seatunnel-starter.jar:2.3.1] at com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:123) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex.getExecutionState(PhysicalVertex.java:470) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.dag.physical.SubPlan.cancelTask(SubPlan.java:405) ~[seatunnel-starter.jar:2.3.1] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_352] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_352] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_352] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_352] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_352] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_352] at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) ~[?:1.8.0_352] at org.apache.seatunnel.engine.server.dag.physical.SubPlan.cancelPipelineTasks(SubPlan.java:381) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.dag.physical.SubPlan.cancelPipeline(SubPlan.java:367) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.dag.physical.SubPlan.handleCheckpointError(SubPlan.java:555) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.master.JobMaster.lambda$handleCheckpointError$2(JobMaster.java:328) ~[seatunnel-starter.jar:2.3.1] at java.util.ArrayList.forEach(ArrayList.java:1259) ~[?:1.8.0_352] at org.apache.seatunnel.engine.server.master.JobMaster.handleCheckpointError(JobMaster.java:325) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.handleCheckpointError(CheckpointManager.java:175) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:221) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:211) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$null$7(CheckpointCoordinator.java:382) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_352] at org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture.internalCompleteExceptionally(PassiveCompletableFuture.java:74) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture.lambda$new$0(PassiveCompletableFuture.java:31) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_352] at org.apache.seatunnel.engine.server.checkpoint.PendingCheckpoint.abortCheckpoint(PendingCheckpoint.java:173) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$cleanPendingCheckpoint$19(CheckpointCoordinator.java:550) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.ConcurrentHashMap$ValuesView.forEach(ConcurrentHashMap.java:4705) ~[?:1.8.0_352] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.cleanPendingCheckpoint(CheckpointCoordinator.java:548) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:216) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:211) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$null$7(CheckpointCoordinator.java:382) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_352] at org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture.internalCompleteExceptionally(PassiveCompletableFuture.java:74) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture.lambda$new$0(PassiveCompletableFuture.java:31) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_352] at org.apache.seatunnel.engine.server.checkpoint.PendingCheckpoint.abortCheckpoint(PendingCheckpoint.java:173) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$cleanPendingCheckpoint$19(CheckpointCoordinator.java:550) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.ConcurrentHashMap$ValuesView.forEach(ConcurrentHashMap.java:4705) ~[?:1.8.0_352] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.cleanPendingCheckpoint(CheckpointCoordinator.java:548) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:216) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:211) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$null$7(CheckpointCoordinator.java:382) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_352] at org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture.internalCompleteExceptionally(PassiveCompletableFuture.java:74) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture.lambda$new$0(PassiveCompletableFuture.java:31) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_352] at org.apache.seatunnel.engine.server.checkpoint.PendingCheckpoint.abortCheckpoint(PendingCheckpoint.java:173) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$cleanPendingCheckpoint$19(CheckpointCoordinator.java:550) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.ConcurrentHashMap$ValuesView.forEach(ConcurrentHashMap.java:4705) ~[?:1.8.0_352] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.cleanPendingCheckpoint(CheckpointCoordinator.java:548) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:216) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:211) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$null$7(CheckpointCoordinator.java:382) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_352] at org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture.internalCompleteExceptionally(PassiveCompletableFuture.java:74) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.common.utils.PassiveCompletableFuture.lambda$new$0(PassiveCompletableFuture.java:31) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_352] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_352] at org.apache.seatunnel.engine.server.checkpoint.PendingCheckpoint.abortCheckpoint(PendingCheckpoint.java:173) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$cleanPendingCheckpoint$19(CheckpointCoordinator.java:550) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.ConcurrentHashMap$ValuesView.forEach(ConcurrentHashMap.java:4705) ~[?:1.8.0_352] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.cleanPendingCheckpoint(CheckpointCoordinator.java:548) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:216) ~[seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$null$9(CheckpointCoordinator.java:436) ~[seatunnel-starter.jar:2.3.1] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_352] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_352] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_352] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_352] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_352] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_352] at java.lang.Thread.run(Thread.java:750) [?:1.8.0_352] 2023-05-26 16:23:00,834 INFO com.hazelcast.internal.server.tcp.TcpServerConnection - [bigdata07]:5801 [seatunnel_mycluster] [5.1] Initialized new cluster connection between /[10.6.1.219:5801](http://10.6.1.219:5801/) and /[10.6.1.16:47142](http://10.6.1.16:47142/) 2023-05-26 16:23:01,223 INFO com.hazelcast.internal.diagnostics.HealthMonitor - [bigdata07]:5801 [seatunnel_mycluster] [5.1] processors=16, physical.memory.total=123.8G, physical.memory.free=2.4G, swap.space.total=0, swap.space.free=0, heap.memory.used=1.7G, heap.memory.free=322.9M, heap.memory.total=2.0G, heap.memory.max=2.0G, heap.memory.used/total=84.21%, heap.memory.used/max=84.21%, minor.gc.count=20828, minor.gc.time=137944ms, major.gc.count=5, major.gc.time=2090ms, load.process=4.34%, load.system=7.67%, load.systemAverage=1.80, thread.count=224, thread.peakCount=236, cluster.timeDiff=0, event.q.size=0, executor.q.async.size=0, executor.q.client.size=0, executor.q.client.query.size=0, executor.q.client.blocking.size=0, executor.q.query.size=0, executor.q.scheduled.size=0, executor.q.io.size=0, executor.q.system.size=0, executor.q.operations.size=0, executor.q.priorityOperation.size=0, operations.completed.count=1134122, executor.q.mapLoad.size=0, executor.q.mapLoadAllKeys.si ze=0, executor.q.cluster.size=0, executor.q.response.size=0, operations.running.count=0, operations.pending.invocations.percentage=0.00%, operations.pending.invocations.count=0, proxy.count=41, clientEndpoint.count=2, connection.active.count=0, client.connection.count=0, connection.count=0 ``` ### Flink or Spark Version _No response_ ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
