2023-02-24 16:05:26,104 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph
[] - Source: Custom Source (1/1)
(9644f5bc58ed204c6d31bb133156fa95_bc764cd8ddf7a0cff126f51c16239658_0_0)
switched from RUNNING to CANCELING.
2023-02-24 16:05:26,108 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph
[] - Flat Map (1/2)
(9644f5bc58ed204c6d31bb133156fa95_0a448493b4782967b150582570326227_0_0)
switched from RUNNING to CANCELING.
2023-02-24 16:05:26,108 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph
[] - Flat Map (2/2)
(9644f5bc58ed204c6d31bb133156fa95_0a448493b4782967b150582570326227_1_0)
switched from RUNNING to CANCELING.
2023-02-24 16:05:26,108 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph
[] - KeyedProcess -> Sink: Unnamed (2/2)
(9644f5bc58ed204c6d31bb133156fa95_e70bbd798b564e0a50e10e343f1ac56b_1_0)
switched from RUNNING to CANCELING.
2023-02-24 16:05:26,137 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph
[] - Flat Map (1/2)
(9644f5bc58ed204c6d31bb133156fa95_0a448493b4782967b150582570326227_0_0)
switched from CANCELING to CANCELED.
2023-02-24 16:05:26,140 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph
[] - Flat Map (2/2)
(9644f5bc58ed204c6d31bb133156fa95_0a448493b4782967b150582570326227_1_0)
switched from CANCELING to CANCELED.
2023-02-24 16:05:26,141 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph
[] - KeyedProcess -> Sink: Unnamed (2/2)
(9644f5bc58ed204c6d31bb133156fa95_e70bbd798b564e0a50e10e343f1ac56b_1_0)
switched from CANCELING to CANCELED.
2023-02-24 16:05:26,145 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Received resource requirements from job 299ff4a6ea193d7c110a19a3de33edab:
[ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN},
numberOfRequiredSlots=1}]
2023-02-24 16:05:26,152 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph
[] - Source: Custom Source (1/1)
(9644f5bc58ed204c6d31bb133156fa95_bc764cd8ddf7a0cff126f51c16239658_0_0)
switched from CANCELING to CANCELED.
2023-02-24 16:05:26,155 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Clearing resource requirements of job 299ff4a6ea193d7c110a19a3de33edab
2023-02-24 16:05:26,154 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph
[] - Job [10204-08]BatchRunningTimesLBSHBaseJobApp
(299ff4a6ea193d7c110a19a3de33edab) switched from state FAILING to FAILED.
org.apache.flink.runtime.JobException: Recovery is suppressed by
NoRestartBackoffTimeStrategy
at
org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:139)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:83)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.scheduler.DefaultScheduler.recordTaskFailure(DefaultScheduler.java:256)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:247)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.scheduler.DefaultScheduler.onTaskFailed(DefaultScheduler.java:240)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.scheduler.SchedulerBase.onTaskExecutionStateUpdate(SchedulerBase.java:738)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:715)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:78)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:477)
~[flink-runtime-1.16.0.jar:1.16.0]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:?]
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:?]
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:309)
~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at
org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:307)
~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:222)
~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:84)
~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:168)
~[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.actor.Actor.aroundReceive(Actor.scala:537)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.actor.Actor.aroundReceive$(Actor.scala:535)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.actor.ActorCell.invoke(ActorCell.scala:548)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.dispatch.Mailbox.run(Mailbox.scala:231)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
[flink-rpc-akka_24401257-5903-4217-89f4-09e8c9e6f005.jar:1.16.0]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) [?:?]
at
java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
[?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
[?:?]
at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
[?:?]
Caused by: org.apache.flink.runtime.taskmanager.AsynchronousException: Caught
exception while processing timer.
at
org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1568)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1543)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1683)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$22(StreamTask.java:1672)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:383)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:345)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
~[flink-runtime-1.16.0.jar:1.16.0]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
Caused by: org.apache.flink.streaming.runtime.tasks.TimerException:
org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException:
Could not forward element to next operator
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1683)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$22(StreamTask.java:1672)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:383)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:345)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
~[flink-runtime-1.16.0.jar:1.16.0]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
Caused by:
org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException:
Could not forward element to next operator
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:99)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.transformation(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:87)
~[?:?]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.resultOut(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:76)
~[?:?]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimerCallback(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:48)
~[?:?]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimer(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:43)
~[?:?]
at
org.apache.flink.streaming.api.operators.KeyedProcessOperator.invokeUserFunction(KeyedProcessOperator.java:91)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.api.operators.KeyedProcessOperator.onProcessingTime(KeyedProcessOperator.java:76)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.api.operators.InternalTimerServiceImpl.onProcessingTime(InternalTimerServiceImpl.java:284)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1681)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$22(StreamTask.java:1672)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:383)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:345)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
~[flink-runtime-1.16.0.jar:1.16.0]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
Caused by: java.lang.RuntimeException: exception
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.checkErrorAndRethrow(BatchRunningTimesHBaseUdfRichSinkFunction.java:264)
~[?:?]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.flush(BatchRunningTimesHBaseUdfRichSinkFunction.java:257)
~[?:?]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.invoke(BatchRunningTimesHBaseUdfRichSinkFunction.java:235)
~[?:?]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.invoke(BatchRunningTimesHBaseUdfRichSinkFunction.java:37)
~[?:?]
at
org.apache.flink.streaming.api.operators.StreamSink.processElement(StreamSink.java:54)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:82)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.transformation(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:87)
~[?:?]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.resultOut(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:76)
~[?:?]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimerCallback(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:48)
~[?:?]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimer(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:43)
~[?:?]
at
org.apache.flink.streaming.api.operators.KeyedProcessOperator.invokeUserFunction(KeyedProcessOperator.java:91)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.api.operators.KeyedProcessOperator.onProcessingTime(KeyedProcessOperator.java:76)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.api.operators.InternalTimerServiceImpl.onProcessingTime(InternalTimerServiceImpl.java:284)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1681)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$22(StreamTask.java:1672)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:383)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:345)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
~[flink-streaming-java-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
~[flink-runtime-1.16.0.jar:1.16.0]
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
~[flink-runtime-1.16.0.jar:1.16.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
~[flink-runtime-1.16.0.jar:1.16.0]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
Caused by: java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
~[?:?]
at java.util.ArrayList$Itr.next(ArrayList.java:997) ~[?:?]
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:189)
~[hbase-client-2.2.2.jar:2.2.2]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.flush(BatchRunningTimesHBaseUdfRichSinkFunction.java:244)
~[?:?]
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.lambda$open$0(BatchRunningTimesHBaseUdfRichSinkFunction.java:97)
~[?:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
~[?:?]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
~[?:?]
at java.lang.Thread.run(Thread.java:834) ~[?:?]
2023-02-24 16:05:26,158 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] -
Stopping checkpoint coordinator for job 299ff4a6ea193d7c110a19a3de33edab.
2023-02-24 16:05:26,178 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher []
- Job 299ff4a6ea193d7c110a19a3de33edab reached terminal state FAILED.
org.apache.flink.runtime.JobException: Recovery is suppressed by
NoRestartBackoffTimeStrategy
at
org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:139)
at
org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:83)
at
org.apache.flink.runtime.scheduler.DefaultScheduler.recordTaskFailure(DefaultScheduler.java:256)
at
org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:247)
at
org.apache.flink.runtime.scheduler.DefaultScheduler.onTaskFailed(DefaultScheduler.java:240)
at
org.apache.flink.runtime.scheduler.SchedulerBase.onTaskExecutionStateUpdate(SchedulerBase.java:738)
at
org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:715)
at
org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:78)
at
org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:477)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:309)
at
org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:307)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:222)
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:84)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:168)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at akka.actor.Actor.aroundReceive(Actor.scala:537)
at akka.actor.Actor.aroundReceive$(Actor.scala:535)
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)
at akka.actor.ActorCell.invoke(ActorCell.scala:548)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
at akka.dispatch.Mailbox.run(Mailbox.scala:231)
at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: org.apache.flink.runtime.taskmanager.AsynchronousException: Caught
exception while processing timer.
at
org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1568)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1543)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1683)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$22(StreamTask.java:1672)
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
at
org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398)
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:383)
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:345)
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by:
TimerException{org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException:
Could not forward element to next operator}
... 15 more
Caused by:
org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException:
Could not forward element to next operator
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:99)
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57)
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29)
at
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)
at
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)
at
org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51)
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.transformation(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:87)
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.resultOut(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:76)
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimerCallback(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:48)
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.onTimer(BatchRunningTimesHBaseUdfKeyedProcessFunctionV2.java:43)
at
org.apache.flink.streaming.api.operators.KeyedProcessOperator.invokeUserFunction(KeyedProcessOperator.java:91)
at
org.apache.flink.streaming.api.operators.KeyedProcessOperator.onProcessingTime(KeyedProcessOperator.java:76)
at
org.apache.flink.streaming.api.operators.InternalTimerServiceImpl.onProcessingTime(InternalTimerServiceImpl.java:284)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1681)
... 14 more
Caused by: java.lang.RuntimeException: exception
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.checkErrorAndRethrow(BatchRunningTimesHBaseUdfRichSinkFunction.java:264)
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.flush(BatchRunningTimesHBaseUdfRichSinkFunction.java:257)
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.invoke(BatchRunningTimesHBaseUdfRichSinkFunction.java:235)
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.invoke(BatchRunningTimesHBaseUdfRichSinkFunction.java:37)
at
org.apache.flink.streaming.api.operators.StreamSink.processElement(StreamSink.java:54)
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:82)
... 27 more
Caused by: java.util.ConcurrentModificationException
at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:189)
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.flush(BatchRunningTimesHBaseUdfRichSinkFunction.java:244)
at
com.flink.batch.lbs.running.hbase.udf.BatchRunningTimesHBaseUdfRichSinkFunction.lambda$open$0(BatchRunningTimesHBaseUdfRichSinkFunction.java:97)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
... 1 more
2023-02-24 16:05:26,225 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher []
- Job 299ff4a6ea193d7c110a19a3de33edab has been registered for cleanup in the
JobResultStore after reaching a terminal state.
2023-02-24 16:05:26,231 INFO
org.apache.flink.runtime.jobmaster.JobMaster
[] - Stopping the JobMaster for job
'[10204-08]BatchRunningTimesLBSHBaseJobApp' (299ff4a6ea193d7c110a19a3de33edab).
2023-02-24 16:05:26,237 INFO
org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore [] -
Shutting down
2023-02-24 16:05:26,239 INFO
org.apache.flink.runtime.jobmaster.JobMaster
[] - Disconnect TaskExecutor localhost:45013-1b5bd3
because: Stopping JobMaster for job '[10204-08]BatchRunningTimesLBSHBaseJobApp'
(299ff4a6ea193d7c110a19a3de33edab).
2023-02-24 16:05:26,240 INFO
org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] -
Releasing slot [ce80357b0859df0b268566294786c8a9].
2023-02-24 16:05:26,242 INFO
org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] -
Releasing slot [7f78bde0f41f2cf0931a01a54db98bfb].
2023-02-24 16:05:26,243 INFO
org.apache.flink.runtime.jobmaster.JobMaster
[] - Close ResourceManager connection
4d0adaf4e63930c3693def906b4a3e7e: Stopping JobMaster for job
'[10204-08]BatchRunningTimesLBSHBaseJobApp' (299ff4a6ea193d7c110a19a3de33edab).
2023-02-24 16:05:26,246 INFO
org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] -
Disconnect job manager
00000000000000000000000000000...@akka.tcp://flink@localhost:6123/user/rpc/jobmanager_2
for job 299ff4a6ea193d7c110a19a3de33edab from the resource manager.
??????
1028108...@qq.com
