fhueske commented on a change in pull request #6823: [FLINK-10134] UTF-16
support for TextInputFormat bug refixed
URL: https://github.com/apache/flink/pull/6823#discussion_r225549931
##########
File path:
flink-java/src/main/java/org/apache/flink/api/java/io/TextInputFormat.java
##########
@@ -86,13 +87,13 @@ public void configure(Configuration parameters) {
@Override
public String readRecord(String reusable, byte[] bytes, int offset, int
numBytes) throws IOException {
//Check if \n is used as delimiter and the end of this line is
a \r, then remove \r from the line
- if (this.getDelimiter() != null && this.getDelimiter().length
== 1
- && this.getDelimiter()[0] == NEW_LINE && offset
+ numBytes >= 1
- && bytes[offset + numBytes - 1] ==
CARRIAGE_RETURN){
- numBytes -= 1;
+ if (this.getDelimiter() != null && this.getDelimiter().length
>= 1
Review comment:
I think this should be refactored a bit. Move the logic into a separate
(private static) method and try to simplify it.
We can check outside of `readRecord()` if the delimiter is equal to NEW_LINE
or not. The whole index arithmetic is also not very easy to understand.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
