adzcai opened a new issue, #13405:
URL: https://github.com/apache/arrow/issues/13405
Hello, I've been having some issues with loading a Parquet file with complex
types into Arrow for Java, and I was hoping here would be a good place to ask.
I've written the following table to a Parquet file:
```none
lists: list<item: int8>
child 0, item: int8
structs: struct<x: int64, y: bool>
child 0, x: int64
child 1, y: bool
----
lists: [[[3,4,5],[6,7]]]
structs: [
-- is_valid: all not null
-- child 0 type: int64
[1,3]
-- child 1 type: bool
[false,true]]
```
Then, I'm trying to load this file into a `VectorSchemaRoot` in Java with a
`VectorLoader` using [this
code](https://gist.github.com/adzcai/4e1a9c130341ad8073cb1976174b1301)
according to [the Java Arrow
documentation](https://arrow.apache.org/docs/java/vector_schema_root.html).
However, it breaks with the following error:
```none
Exception in thread "main" java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:864)
at
org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:104)
at
org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:133)
at org.apache.arrow.vector.VectorLoader.load(VectorLoader.java:84)
at Scratch.main(scratch.java:57)
Suppressed: java.lang.IllegalStateException: Memory was leaked by
query. Memory leaked: (41998)
Allocator(ROOT) 0/41998/41998/9223372036854775807 (res/actual/peak/limit)
at
org.apache.arrow.memory.BaseAllocator.close(BaseAllocator.java:437)
at
org.apache.arrow.memory.RootAllocator.close(RootAllocator.java:29)
at Scratch.main(scratch.java:61)
```
I'm not super familiar with Arrow internals, but after some quick debugging,
it seems that in some places, a List field takes up two buffers, while in other
places it only takes up one. But I could be mistaken. The code works fine for
non-complex types.
Any advice would be greatly appreciated! Thanks for your time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
