Hi,看起来是需要一个脏数据处理的逻辑的,可以在jira上建一个issue,并给出更加详细的报错信息和可能的脏数据信息呀。建立issue后,可以在邮件里回复下。
Asahi Lee 于2022年9月14日周三 09:33写道:
> 是的,运行一段时间后发生错误,null指针的代码哪里是不是要非空处理?
>
>
>
>
> -- 原始邮件 --
> 发件人:
> "user-zh"
> <
> xyzhong...@163.com>;
> 发送时间: 2022年9月9日(星期五) 晚上8:37
> 收件人: "user-zh"
> 主题: Re:这里为什么会报null指针错误,和源表数据有关系吗?
>
>
>
> Hi,看上去是遇到了一条脏数据,问一下是在运行了一段时间之后突然报错的嘛?
>
>
>
>
>
>
>
> --
>
> Best!
> Xuyang
>
>
>
>
>
> At 2022-09-09 11:46:47, "Asahi Lee" >2022-09-09 11:36:42,866 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Source:
> HiveSource-ods_jt_hrs.ods_hrmis_HR_EMPL_Education (1/1)
> (2a68412dab3602a1eeda5a750b308e23) switched from RUNNING to FAILED on
> container_1658144991761_106260_01_02 @ hhny-cdh05 (dataPort=45015).
> >java.lang.RuntimeException: null
> >at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:105)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:89)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:43)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask$AsyncDataOutputToOutput.emitRecord(SourceOperatorStreamTask.java:196)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.api.operators.source.NoOpTimestampsAndWatermarks$TimestampsOnlyOutput.collect(NoOpTimestampsAndWatermarks.java:97)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.api.operators.source.NoOpTimestampsAndWatermarks$TimestampsOnlyOutput.collect(NoOpTimestampsAndWatermarks.java:91)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.connector.file.src.impl.FileSourceRecordEmitter.emitRecord(FileSourceRecordEmitter.java:45)
> ~[flink-table_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.connector.file.src.impl.FileSourceRecordEmitter.emitRecord(FileSourceRecordEmitter.java:35)
> ~[flink-table_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:143)
> ~[flink-table_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:350)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:496)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
> >Caused by: java.lang.NullPointerException
> >at
> org.apache.flink.table.data.writer.BinaryWriter.write(BinaryWriter.java:118)
> ~[flink-table_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.toBinaryRow(RowDataSerializer.java:204)
> ~[flink-table_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.serialize(RowDataSerializer.java:103)
> ~[flink-table_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.serialize(RowDataSerializer.java:48)
> ~[flink-table_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:168)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:46)
> ~[flink-dist_2.11-1.14.3.jar:1.14.3]
> >at
> org.apa