[
https://issues.apache.org/jira/browse/FLINK-10204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16591329#comment-16591329
]
ASF GitHub Bot commented on FLINK-10204:
----------------------------------------
zentol commented on a change in pull request #6610: [FLINK-10204] - fix
serialization/copy error for LatencyMarker records.
URL: https://github.com/apache/flink/pull/6610#discussion_r212558074
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/streamrecord/LatencyMarker.java
##########
@@ -67,31 +67,40 @@ public int getSubtaskIndex() {
//
------------------------------------------------------------------------
@Override
- public boolean equals(Object o) {
- if (this == o) {
+ public boolean equals(Object obj) {
+ if (this == obj) {
return true;
}
- if (o == null || getClass() != o.getClass()){
+ if (obj == null) {
return false;
}
-
- LatencyMarker that = (LatencyMarker) o;
-
- if (markedTime != that.markedTime) {
+ if (getClass() != obj.getClass()) {
return false;
}
- if (operatorId != that.operatorId) {
+ LatencyMarker other = (LatencyMarker) obj;
+ if (markedTime != other.markedTime) {
return false;
}
- return subtaskIndex == that.subtaskIndex;
-
+ if (operatorId == null) {
+ if (other.operatorId != null) {
+ return false;
+ }
+ } else if (!operatorId.equals(other.operatorId)) {
+ return false;
+ }
+ if (subtaskIndex != other.subtaskIndex) {
+ return false;
+ }
+ return true;
}
@Override
public int hashCode() {
- int result = (int) (markedTime ^ (markedTime >>> 32));
Review comment:
seems unrelated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Job is marked as FAILED after serialization exception
> -----------------------------------------------------
>
> Key: FLINK-10204
> URL: https://issues.apache.org/jira/browse/FLINK-10204
> Project: Flink
> Issue Type: Bug
> Reporter: Ben La Monica
> Priority: Major
> Labels: pull-request-available
>
> We have a long running flink job that eventually fails and is shut down due
> to an internal serialization exception that we keep on getting. Here is the
> stack trace:
> {code:java}
> 2018-08-23 18:39:48,199 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph - Job NAV Estimation
> (4b5463d76167f9f5aac83a890e8a867d) switched from state FAILING to FAILED.
> java.io.IOException: Corrupt stream, found tag: 127
> at
> org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.deserialize(StreamElementSerializer.java:219)
> at
> org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.deserialize(StreamElementSerializer.java:49)
> at
> org.apache.flink.runtime.plugable.NonReusingDeserializationDelegate.read(NonReusingDeserializationDelegate.java:55)
> at
> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer.getNextRecord(SpillingAdaptiveSpanningRecordDeserializer.java:140)
> at
> org.apache.flink.streaming.runtime.io.StreamTwoInputProcessor.processInput(StreamTwoInputProcessor.java:206)
> at
> org.apache.flink.streaming.runtime.tasks.TwoInputStreamTask.run(TwoInputStreamTask.java:115)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:306)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
> at java.lang.Thread.run(Thread.java:748){code}
>
> I think I have tracked down the issue to a mismatch in the
> serialization/deserialization/copy code in the StreamElementSerializer with
> regards to the LATENCY_MARKER.
> The Serialization logic writes 3 LONGs and an INT. The copy logic only writes
> (and reads) a LONG and 2 INTs. Adding a test for the LatencyMarker throws an
> EOFException, and fixing the copy code causes the test to pass again.
> I've written a unit test that highlights the problem, and have written the
> code to correct it.
> I'll submit a PR that goes along with it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
