saurabhchauhan006 commented on issue #44311:
URL: https://github.com/apache/arrow/issues/44311#issuecomment-2498314715
So the problem is that arrow 17.0.0 version is compiled using java 9 and
above.
In ArrowFileReader.readSchema() it is calling ByteBuffer.flip() and because
in java 8 there is no implementation for ByteBuffer.flip() it fails.
To fix this I have forked this class and type casted ByteBuffer class with
Buffer.
`@Override
protected Schema readSchema() throws IOException {
if (footer == null) {
if (in.size() <= (ArrowMagic.MAGIC_LENGTH * 2 + 4)) {
throw new InvalidArrowFileException("file too small: " +
in.size());
}
ByteBuffer buffer = ByteBuffer.allocate(4 +
ArrowMagic.MAGIC_LENGTH);
long footerLengthOffset = in.size() - buffer.remaining();
in.setPosition(footerLengthOffset);
in.readFully(buffer);
((Buffer)buffer).flip();
byte[] array = buffer.array();
if (!ArrowMagic.validateMagic(Arrays.copyOfRange(array, 4,
array.length))) {
throw new InvalidArrowFileException(
"missing Magic number " +
Arrays.toString(buffer.array()));
}
int footerLength = MessageSerializer.bytesToInt(array);
if (footerLength <= 0
|| footerLength + ArrowMagic.MAGIC_LENGTH * 2 + 4 >
in.size()
|| footerLength > footerLengthOffset) {
throw new InvalidArrowFileException("invalid footer length:
" + footerLength);
}
long footerOffset = footerLengthOffset - footerLength;
LOGGER.debug("Footer starts at {}, length: {}", footerOffset,
footerLength);
ByteBuffer footerBuffer = ByteBuffer.allocate(footerLength);
in.setPosition(footerOffset);
in.readFully(footerBuffer);
footerBuffer.flip();
Footer footerFB = Footer.getRootAsFooter(footerBuffer);
this.footer = new ArrowFooter(footerFB);
}
MetadataV4UnionChecker.checkRead(footer.getSchema(),
footer.getMetadataVersion());
return footer.getSchema();
}`
So basically in java 9 and above ByteBuffer.flip() just calls Buffer.flip().
So by this way it solves the problem with Java 8.
I am not pushing a fix for this as the current version of Arrow requires
java 11 and above, so there is no point of fixing this now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
