lidavidm commented on a change in pull request #9147:
URL: https://github.com/apache/arrow/pull/9147#discussion_r555809311
##########
File path:
java/flight/flight-core/src/main/java/org/apache/arrow/flight/ArrowMessage.java
##########
@@ -259,7 +259,8 @@ private static ArrowMessage frame(BufferAllocator
allocator, final InputStream s
ArrowBuf body = null;
ArrowBuf appMetadata = null;
while (stream.available() > 0) {
- int tag = readRawVarint32(stream);
+ int tag = readRawVarint32WithEOFCheck(stream);
+
switch (tag) {
Review comment:
That's not true, though, since readRawVarint32WithEOFCheck has a branch
that returns -1.
##########
File path:
java/flight/flight-core/src/main/java/org/apache/arrow/flight/ArrowMessage.java
##########
@@ -332,6 +333,23 @@ private static ArrowMessage frame(BufferAllocator
allocator, final InputStream s
}
+ /**
+ * Get first byte with EOF check, it is especially needed when using grpc
compression.
+ * InflaterInputStream need another read to change reachEOF after all bytes
has been read.
+ *
+ * @param is InputStream
+ * @return -1 if stream is not available, otherwise it will return the
actual value.
+ * @throws IOException Read first byte failed.
+ */
+ private static int readRawVarint32WithEOFCheck(InputStream is) throws
IOException {
+ int firstByte = is.read();
+ if (is.available() <= 0) {
+ return -1;
+ } else {
+ return CodedInputStream.readRawVarint32(firstByte, is);
+ }
+ }
+
private static int readRawVarint32(InputStream is) throws IOException {
int firstByte = is.read();
Review comment:
I'm not sure what you mean - if is.read() returns -1, the stream is at
EOF, so there's never any more data to read.
I also mean, we should consolidate readRawVarint32 and
readRawVarint32WithEOFCheck since there's never any reason to proceed if we're
at EOF, and furthermore, CodedInputStream#readRawVarint32(int, InputStream)
itself doesn't do an EOF check (it assumes the caller has).
##########
File path:
java/flight/flight-core/src/main/java/org/apache/arrow/flight/ArrowMessage.java
##########
@@ -332,6 +333,23 @@ private static ArrowMessage frame(BufferAllocator
allocator, final InputStream s
}
+ /**
+ * Get first byte with EOF check, it is especially needed when using grpc
compression.
+ * InflaterInputStream need another read to change reachEOF after all bytes
has been read.
+ *
+ * @param is InputStream
+ * @return -1 if stream is not available, otherwise it will return the
actual value.
+ * @throws IOException Read first byte failed.
+ */
+ private static int readRawVarint32WithEOFCheck(InputStream is) throws
IOException {
+ int firstByte = is.read();
Review comment:
That would be an incorrect InputStream implementation, since
InputStream.read() == -1 is defined to be EOF - there should be no data left.
##########
File path:
java/flight/flight-core/src/main/java/org/apache/arrow/flight/ArrowMessage.java
##########
@@ -332,6 +333,23 @@ private static ArrowMessage frame(BufferAllocator
allocator, final InputStream s
}
+ /**
+ * Get first byte with EOF check, it is especially needed when using grpc
compression.
+ * InflaterInputStream need another read to change reachEOF after all bytes
has been read.
+ *
+ * @param is InputStream
+ * @return -1 if stream is not available, otherwise it will return the
actual value.
+ * @throws IOException Read first byte failed.
+ */
+ private static int readRawVarint32WithEOFCheck(InputStream is) throws
IOException {
+ int firstByte = is.read();
Review comment:
Furthermore, available() is defined to be an estimate; available() == 0
doesn't mean EOF.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
