zxylmqx commented on issue #4838:
URL: https://github.com/apache/seatunnel/issues/4838#issuecomment-1566723445
已经指定了不同的[sink.label.prefix]，而且这个是传入的变量，保证每次启动都是唯一的，目前将checkpoint从hdfs变成了locol，错误变成了
2023-05-29 15:07:42,928 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] turn from state DEPLOYING to RUNNING.
2023-05-29 15:07:43,010 ERROR
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - task
running failed
java.util.concurrent.CompletionException:
org.apache.seatunnel.common.utils.SeaTunnelException: java.io.EOFException
at
com.hazelcast.spi.impl.AbstractInvocationFuture.returnOrThrowWithJoinConventions(AbstractInvocationFuture.java:819)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.AbstractInvocationFuture.resolveAndThrowWithJoinConvention(AbstractInvocationFuture.java:835)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.AbstractInvocationFuture.join(AbstractInvocationFuture.java:553)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.restoreTaskState(CheckpointCoordinator.java:250)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$reportedTask$1(CheckpointCoordinator.java:189)
~[seatunnel-starter.jar:2.3.1]
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
~[?:1.8.0_352]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_352]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_352]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_352]
Caused by: org.apache.seatunnel.common.utils.SeaTunnelException:
java.io.EOFException
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$run$0(NotifyTaskRestoreOperation.java:100)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.run(NotifyTaskRestoreOperation.java:85)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
~[seatunnel-starter.jar:2.3.1]
Caused by: java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
~[?:1.8.0_352]
at java.io.DataInputStream.readLong(DataInputStream.java:416)
~[?:1.8.0_352]
at
org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkStateSerializer.deserialize(DorisSinkStateSerializer.java:45)
~[?:?]
at
org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkStateSerializer.deserialize(DorisSinkStateSerializer.java:29)
~[?:?]
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.lambda$null$2(SinkFlowLifeCycle.java:239)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.common.utils.ExceptionUtil.sneaky(ExceptionUtil.java:138)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.lambda$restoreState$3(SinkFlowLifeCycle.java:235)
~[seatunnel-starter.jar:2.3.1]
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
~[?:1.8.0_352]
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
~[?:1.8.0_352]
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
~[?:1.8.0_352]
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
~[?:1.8.0_352]
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:240)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$14(SeaTunnelTask.java:383)
~[seatunnel-starter.jar:2.3.1]
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
~[?:1.8.0_352]
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
~[?:1.8.0_352]
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
~[?:1.8.0_352]
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
~[?:1.8.0_352]
at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:380)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$run$0(NotifyTaskRestoreOperation.java:96)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.run(NotifyTaskRestoreOperation.java:85)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
~[seatunnel-starter.jar:2.3.1]
2023-05-29 15:07:43,010 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start
clean pending checkpoint cause CheckpointCoordinator inside have error.
2023-05-29 15:07:43,010 WARN
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] checkpoint have error, cancel the
pipeline
2023-05-29 15:07:43,011 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] turn from state RUNNING to CANCELING.
2023-05-29 15:07:43,014 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn from state RUNNING to
CANCELING.
2023-05-29 15:07:43,014 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn from state RUNNING to
CANCELING.
2023-05-29 15:07:43,015 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] operator to member
[bigdata07]:5801
2023-05-29 15:07:43,015 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn from state RUNNING to
CANCELING.
2023-05-29 15:07:43,018 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] operator to member
[bigdata07]:5801
2023-05-29 15:07:43,019 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] operator to member
[bigdata07]:5801
2023-05-29 15:07:43,019 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn to end state CANCELED.
2023-05-29 15:07:43,019 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn to end state CANCELED.
2023-05-29 15:07:43,019 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] end with state CANCELED
2023-05-29 15:07:43,019 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] end with state CANCELED
2023-05-29 15:07:43,021 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn to end state CANCELED.
2023-05-29 15:07:43,021 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] end with state CANCELED
2023-05-29 15:07:43,022 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] end with state FAILED
2023-05-29 15:07:43,022 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Can restore
pipeline Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)]
2023-05-29 15:07:43,022 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Restore time 2,
pipeline Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)]
2023-05-29 15:07:43,023 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn to state CREATED.
2023-05-29 15:07:43,024 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn to state CREATED.
2023-05-29 15:07:43,024 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn to state CREATED.
2023-05-29 15:07:43,025 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] is in state CREATED when init
state future
2023-05-29 15:07:43,025 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] is in state CREATED when init
state future
2023-05-29 15:07:43,025 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] is in state CREATED when
init state future
2023-05-29 15:07:43,025 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - received
restore CheckpointCoordinator with alreadyStarted= false
2023-05-29 15:07:43,025 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start
clean pending checkpoint cause CheckpointCoordinator reset.
2023-05-29 15:07:43,025 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] turn from state CREATED to SCHEDULED.
2023-05-29 15:07:43,026 INFO
org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager -
received slot active check success, profile:
SlotProfile{worker=[bigdata07]:5801, slotID=1, ownerJobID=715450079284035585,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'}
2023-05-29 15:07:43,026 INFO
org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler - use active
old profile: SlotProfile{worker=[bigdata07]:5801, slotID=1,
ownerJobID=715450079284035585, assigned=true,
resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}},
sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'} for task Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)]
2023-05-29 15:07:43,026 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn from state CREATED to
SCHEDULED.
2023-05-29 15:07:43,027 INFO
org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager -
received slot active check success, profile:
SlotProfile{worker=[bigdata07]:5801, slotID=2, ownerJobID=715450079284035585,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'}
2023-05-29 15:07:43,027 INFO
org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler - use active
old profile: SlotProfile{worker=[bigdata07]:5801, slotID=2,
ownerJobID=715450079284035585, assigned=true,
resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}},
sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'} for task Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)]
2023-05-29 15:07:43,028 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn from state CREATED to
SCHEDULED.
2023-05-29 15:07:43,028 INFO
org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager -
received slot active check success, profile:
SlotProfile{worker=[bigdata07]:5801, slotID=3, ownerJobID=715450079284035585,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'}
2023-05-29 15:07:43,028 INFO
org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler - use active
old profile: SlotProfile{worker=[bigdata07]:5801, slotID=3,
ownerJobID=715450079284035585, assigned=true,
resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}},
sequence='0c2626d6-416f-4f79-9e44-2e98873f8751'} for task Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)]
2023-05-29 15:07:43,028 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn from state CREATED to
SCHEDULED.
2023-05-29 15:07:43,029 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] turn from state SCHEDULED to DEPLOYING.
2023-05-29 15:07:43,030 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn from state SCHEDULED
to DEPLOYING.
2023-05-29 15:07:43,031 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn from state SCHEDULED to
DEPLOYING.
2023-05-29 15:07:43,032 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn from state SCHEDULED to
DEPLOYING.
2023-05-29 15:07:43,066 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn from state DEPLOYING to
RUNNING.
2023-05-29 15:07:43,073 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn from state DEPLOYING
to RUNNING.
2023-05-29 15:07:43,074 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn from state DEPLOYING to
RUNNING.
2023-05-29 15:07:43,074 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] turn from state DEPLOYING to RUNNING.
2023-05-29 15:07:43,151 ERROR
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - task
running failed
java.util.concurrent.CompletionException:
org.apache.seatunnel.common.utils.SeaTunnelException: java.io.EOFException
at
com.hazelcast.spi.impl.AbstractInvocationFuture.returnOrThrowWithJoinConventions(AbstractInvocationFuture.java:819)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.AbstractInvocationFuture.resolveAndThrowWithJoinConvention(AbstractInvocationFuture.java:835)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.AbstractInvocationFuture.join(AbstractInvocationFuture.java:553)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.restoreTaskState(CheckpointCoordinator.java:250)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$reportedTask$1(CheckpointCoordinator.java:189)
~[seatunnel-starter.jar:2.3.1]
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
~[?:1.8.0_352]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_352]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_352]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_352]
Caused by: org.apache.seatunnel.common.utils.SeaTunnelException:
java.io.EOFException
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$run$0(NotifyTaskRestoreOperation.java:100)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.run(NotifyTaskRestoreOperation.java:85)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
~[seatunnel-starter.jar:2.3.1]
Caused by: java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
~[?:1.8.0_352]
at java.io.DataInputStream.readLong(DataInputStream.java:416)
~[?:1.8.0_352]
at
org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkStateSerializer.deserialize(DorisSinkStateSerializer.java:45)
~[?:?]
at
org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkStateSerializer.deserialize(DorisSinkStateSerializer.java:29)
~[?:?]
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.lambda$null$2(SinkFlowLifeCycle.java:239)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.common.utils.ExceptionUtil.sneaky(ExceptionUtil.java:138)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.lambda$restoreState$3(SinkFlowLifeCycle.java:235)
~[seatunnel-starter.jar:2.3.1]
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
~[?:1.8.0_352]
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
~[?:1.8.0_352]
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
~[?:1.8.0_352]
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
~[?:1.8.0_352]
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:240)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$14(SeaTunnelTask.java:383)
~[seatunnel-starter.jar:2.3.1]
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
~[?:1.8.0_352]
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
~[?:1.8.0_352]
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
~[?:1.8.0_352]
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
~[?:1.8.0_352]
at
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
~[?:1.8.0_352]
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
~[?:1.8.0_352]
at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:380)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$run$0(NotifyTaskRestoreOperation.java:96)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48)
~[seatunnel-starter.jar:2.3.1]
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.run(NotifyTaskRestoreOperation.java:85)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
~[seatunnel-starter.jar:2.3.1]
at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
~[seatunnel-starter.jar:2.3.1]
2023-05-29 15:07:43,151 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start
clean pending checkpoint cause CheckpointCoordinator inside have error.
2023-05-29 15:07:43,152 WARN
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] checkpoint have error, cancel the
pipeline
2023-05-29 15:07:43,153 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] turn from state RUNNING to CANCELING.
2023-05-29 15:07:43,156 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn from state RUNNING to
CANCELING.
2023-05-29 15:07:43,157 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn from state RUNNING to
CANCELING.
2023-05-29 15:07:43,158 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn from state RUNNING to
CANCELING.
2023-05-29 15:07:43,160 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] operator to member
[bigdata07]:5801
2023-05-29 15:07:43,160 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] operator to member
[bigdata07]:5801
2023-05-29 15:07:43,160 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] operator to member
[bigdata07]:5801
2023-05-29 15:07:43,161 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] turn to end state CANCELED.
2023-05-29 15:07:43,161 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (2/2)] end with state CANCELED
2023-05-29 15:07:43,161 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] turn to end state CANCELED.
2023-05-29 15:07:43,161 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SplitEnumerator (1/1)] end with state CANCELED
2023-05-29 15:07:43,162 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] turn to end state CANCELED.
2023-05-29 15:07:43,162 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-MySQL-CDC-default]-SourceTask (1/2)] end with state CANCELED
2023-05-29 15:07:43,193 INFO
org.apache.seatunnel.engine.server.master.JobMaster - release the pipeline Job
cus_basic_applet (715450079284035585), Pipeline: [(1/1)] resource
2023-05-29 15:07:43,197 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] end with state FAILED
2023-05-29 15:07:43,197 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - cancel job Job
cus_basic_applet (715450079284035585) because makeJobEndWhenPipelineEnded is
true
2023-05-29 15:07:43,199 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job
cus_basic_applet (715450079284035585) turn from state RUNNING to CANCELLING.
2023-05-29 15:07:43,200 WARN
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job cus_basic_applet
(715450079284035585), Pipeline: [(1/1)] is in end state FAILED, can not be
cancel
2023-05-29 15:07:43,201 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job
cus_basic_applet (715450079284035585) turn from state CANCELLING to FAILING.
2023-05-29 15:07:43,202 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job
cus_basic_applet (715450079284035585) turn from state FAILING to FAILED.
2023-05-29 15:07:43,224 INFO
com.hazelcast.internal.server.tcp.TcpServerConnection - [bigdata05]:5801
[seatunnel_mycluster] [5.1] Connection[id=6, /10.6.1.16:5801->/10.6.1.16:45233,
qualifier=null, endpoint=[10.6.1.16]:45233,
remoteUuid=9b983f15-396a-4d6e-a9bf-2fb02eb9db82, alive=false,
connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the
other side
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
