X-czh opened a new issue, #2204: URL: https://github.com/apache/fury/issues/2204
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/fury/issues) and found no similar issues. ### Version - Fury: 0.10.2 & 0.11.0 & master - JDK: 8u442 ### Component(s) Java ### Minimal reproduce step ```java Fury fury = Fury.builder().withLanguage(Language.JAVA).withCodegen(true).build(); fury.register(ThriftObject.class); ThriftObject o = new ThriftObject(); // The default initial buffer size of 4096 cannot reproduce fury.deserialize(new FuryInputStream(new ByteArrayInputStream(fury.serialize(o)), 2)); ``` ### What did you expect to see? No exception. ### What did you see instead? ``` org.apache.fury.exception.DeserializationException: Deserialize failed, read objects are: [VidsAndFlightHitRules(vid:null, rules:null)] at org.apache.fury.util.ExceptionUtils.handleReadFailed(ExceptionUtils.java:63) at org.apache.fury.Fury.deserialize(Fury.java:810) at org.apache.fury.Fury.deserialize(Fury.java:829) at org.apache.fury.Fury.deserialize(Fury.java:822) at com.bytedance.ad.abtest.etl.realtime.util.FuryTest.testFury(FuryTest.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69) at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38) at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:231) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55) Caused by: java.io.IOException: org.apache.thrift.transport.TTransportException: Cannot read. Remote side has closed. Tried to read 1 bytes, but only got 0 bytes. (This is often indicative of an internal error on the server side. Please check your server logs.) at com.bytedance.ad.abtest.meta.thrift.VidsAndFlightHitRules.readObject(VidsAndFlightHitRules.java:428) at org.apache.fury.serializer.ObjectStreamSerializer.read(ObjectStreamSerializer.java:192) at org.apache.fury.Fury.readDataInternal(Fury.java:990) at org.apache.fury.Fury.readRef(Fury.java:874) at org.apache.fury.Fury.deserialize(Fury.java:806) ... 27 more Caused by: org.apache.thrift.transport.TTransportException: Cannot read. Remote side has closed. Tried to read 1 bytes, but only got 0 bytes. (This is often indicative of an internal error on the server side. Please check your server logs.) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:88) at org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:634) at org.apache.thrift.protocol.TCompactProtocol.readFieldBegin(TCompactProtocol.java:539) at com.bytedance.ad.abtest.meta.thrift.VidsAndFlightHitRules$VidsAndFlightHitRulesStandardScheme.read(VidsAndFlightHitRules.java:445) at com.bytedance.ad.abtest.meta.thrift.VidsAndFlightHitRules$VidsAndFlightHitRulesStandardScheme.read(VidsAndFlightHitRules.java:438) at com.bytedance.ad.abtest.meta.thrift.VidsAndFlightHitRules.read(VidsAndFlightHitRules.java:380) at com.bytedance.ad.abtest.meta.thrift.VidsAndFlightHitRules.readObject(VidsAndFlightHitRules.java:426) ... 31 more ``` ### Anything Else? ## RCA 1. A Thrift object implements custom `readObject` and `writeObject` methods, which thereby is handled by Fury's `ObjectStreamSerializer` for compatibility. 2. Thrift's `readObject` method will internally call https://github.com/apache/thrift/blob/df626d768a87fe07fef215b4dde831185e6929d7/lib/javame/src/org/apache/thrift/transport/TTransport.java#L84 to read bytes: ```java public int readAll(byte[] buf, int off, int len) throws TTransportException { int got = 0; int ret = 0; while (got < len) { ret = read(buf, off+got, len-got); if (ret <= 0) { throw new TTransportException("Cannot read. Remote side has closed. Tried to read " + len + " bytes, but only got " + got + " bytes."); } got += ret; } return got; } ``` 3. Since Fury overwrites the `ObjectInputStream` impl, the read call internally calls https://github.com/apache/fury/blob/dd3edef18cf986d225600e0669d1cebd2fb0c8d7/java/fury-core/src/main/java/org/apache/fury/serializer/ObjectStreamSerializer.java#L897: ```java public int read(byte[] buf, int offset, int length) throws IOException { if (buf == null) { throw new NullPointerException(); } int endOffset = offset + length; if (offset < 0 || length < 0 || endOffset > buf.length || endOffset < 0) { throw new IndexOutOfBoundsException(); } int remaining = buffer.remaining(); if (remaining < length) { buffer.readBytes(buf, offset, remaining); return remaining; } else { buffer.readBytes(buf, offset, length); return length; } } ``` When there happen to be 0 remaining bytes in the buffer, the call returns 0, and Thrift complains about it with a `TTransportException`. 4. From the comments of the [`ObjectInputStream#read`](https://github.com/openjdk/jdk/blob/e09d2e275bc646201a8da39bd4b977d3fda97954/src/java.base/share/classes/java/io/InputStream.java#L283), it only reads up to `len` bytes, but a smaller number may be read. However, Thrift does not accept reading 0 bytes and the JDK's impl will never return 0 when len != 0 too. Not sure if this should be considered a bug on the Thrift side or on the Fury side. ## Potential Fix Change the impl of `FuryObjectInputStream#read` s.t. it never returns 0 when len != 0 as well. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
