zhijiangW commented on a change in pull request #6705: [FLINK-10356][network]
add sanity checks to SpillingAdaptiveSpanningRecordDeserializer
URL: https://github.com/apache/flink/pull/6705#discussion_r225395730
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/SpillingAdaptiveSpanningRecordDeserializer.java
##########
@@ -137,7 +162,16 @@ else if (remaining == 0) {
// spanning record case
if (this.spanningWrapper.hasFullRecord()) {
// get the full record
- target.read(this.spanningWrapper.getInputView());
+ try {
+
target.read(this.spanningWrapper.getInputView());
+ } catch (EOFException e) {
+ Optional<String> deserializationError =
this.spanningWrapper.getDeserializationError(1);
Review comment:
I do not quite understand why we set `addToReadBytes` as 1 here.
If the `target.read` is successful, then we do
`spanningWrapper.getDeserializationError(0)` in the following
`moveRemainderToNonSpanningDeserializer` and it makes sense, otherwise we do
`spanningWrapper.getDeserializationError(1)`.
The `spanningWrapper.getDeserializationError(0)` may be also suitable for
the exception case? Because we only want to show some internal informations
during exceptions for debugging. Then we just need one check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
