[
https://issues.apache.org/jira/browse/ARROW-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haowei Yu updated ARROW-9661:
-----------------------------
Description:
I have some bytes, which are in IPC streaming format, but Java failed to read
data and give me back error
{code:java}
java.io.IOException: Expected schema but header was 0
{code}
Here is my java code
{code:java}
BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);
ArrayList<ValueVector> valueVectors = new ArrayList<>();
final ArrayList<List<ValueVector>> batchOfVectors = new ArrayList<>();
final RootAllocator rootAllocator = new RootAllocator();
InputStream inputStream = new FileInputStream("/tmp/arrowdata");
try (ArrowStreamReader reader = new ArrowStreamReader(inputStream, allocator);
) {
VectorSchemaRoot root = reader.getVectorSchemaRoot();
while (reader.loadNextBatch()) {
valueVectors = new ArrayList<>();
for (FieldVector f : root.getFieldVectors())
{
// transfer will not copy data but transfer ownership of memory
// from streamReader to resultChunk
TransferPair t = f.getTransferPair(rootAllocator);
t.transfer();
valueVectors.add(t.getTo());
}
batchOfVectors.add(valueVectors);
root.clear();
}
}{code}
I am using arrow-vector 1.0.0 and I put my arrow data in the attachment. I am
pretty sure data is correct since I can read data using JS sdk.
I am willing to provide more information if you want. ( Not sure if this is a
netty thing since I do have another version of netty as dependency in my
project )
was:
I have some bytes, which are in IPC streaming format, but Java failed to read
data and give me back error
```
java.io.IOException: Expected schema but header was 0
```
Here is my java code
```
BufferAllocator allocator = new RootAllocator(Long._MAX_VALUE_);
ArrayList<ValueVector> valueVectors = new ArrayList<>();
final ArrayList<List<ValueVector>> batchOfVectors = new ArrayList<>();
final RootAllocator rootAllocator = new RootAllocator();
InputStream inputStream = new FileInputStream("/tmp/arrowdata");
try (ArrowStreamReader reader = new ArrowStreamReader(inputStream, allocator);
) {
VectorSchemaRoot root = reader.getVectorSchemaRoot();
while (reader.loadNextBatch()) {
valueVectors = new ArrayList<>();
for (FieldVector f : root.getFieldVectors()) {
// transfer will not copy data but transfer ownership of memory
// from streamReader to resultChunk
TransferPair t = f.getTransferPair(rootAllocator);
t.transfer();
valueVectors.add(t.getTo());
}
batchOfVectors.add(valueVectors);
root.clear();
}
}
```
> [Java] Failed to read arrow IPC streaming format
> ------------------------------------------------
>
> Key: ARROW-9661
> URL: https://issues.apache.org/jira/browse/ARROW-9661
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Java
> Affects Versions: 1.0.0
> Reporter: Haowei Yu
> Priority: Major
>
> I have some bytes, which are in IPC streaming format, but Java failed to read
> data and give me back error
> {code:java}
> java.io.IOException: Expected schema but header was 0
> {code}
> Here is my java code
> {code:java}
> BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);
> ArrayList<ValueVector> valueVectors = new ArrayList<>();
> final ArrayList<List<ValueVector>> batchOfVectors = new ArrayList<>();
> final RootAllocator rootAllocator = new RootAllocator();
> InputStream inputStream = new FileInputStream("/tmp/arrowdata");
> try (ArrowStreamReader reader = new ArrowStreamReader(inputStream,
> allocator); ) {
> VectorSchemaRoot root = reader.getVectorSchemaRoot();
> while (reader.loadNextBatch()) {
> valueVectors = new ArrayList<>();
> for (FieldVector f : root.getFieldVectors())
> {
> // transfer will not copy data but transfer ownership of memory
> // from streamReader to resultChunk
> TransferPair t = f.getTransferPair(rootAllocator);
> t.transfer();
> valueVectors.add(t.getTo());
> }
> batchOfVectors.add(valueVectors);
> root.clear();
> }
> }{code}
> I am using arrow-vector 1.0.0 and I put my arrow data in the attachment. I am
> pretty sure data is correct since I can read data using JS sdk.
> I am willing to provide more information if you want. ( Not sure if this is a
> netty thing since I do have another version of netty as dependency in my
> project )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
