lidavidm commented on code in PR #14506:
URL: https://github.com/apache/arrow/pull/14506#discussion_r1004954569
##########
java/c/src/main/java/org/apache/arrow/c/ArrayImporter.java:
##########
@@ -138,16 +139,143 @@ private List<ArrowBuf> importBuffers(ArrowArray.Snapshot
snapshot) {
buffersCount, vector.getField().getType().getTypeID(), buffers.length);
List<ArrowBuf> result = new ArrayList<>(buffersCount);
+ int bufferIndex = 0;
for (long bufferPtr : buffers) {
ArrowBuf buffer = null;
if (bufferPtr != NULL) {
// See ARROW-17720: [Java] C data interface: Add API to compute
imported buffer size
- int capacity = Integer.MAX_VALUE;
- buffer = new ArrowBuf(referenceManager, null, capacity, bufferPtr);
+ int capacity = vector.getField().getType().accept(new
IntegerArrowTypeVisitor(fieldNode, bufferIndex++));
+ AllocationManager allocationManager = new
CDataAllocationManager(allocator, underlyingAllocation, bufferPtr, capacity);
+ allocator.forceAllocate(capacity);
+ BufferLedger ledger = new BufferLedger(allocator, allocationManager);
+ buffer = new ArrowBuf(ledger, null, capacity, bufferPtr);
buffer.writerIndex(capacity);
}
result.add(buffer);
}
return result;
}
+
+ private static class IntegerArrowTypeVisitor implements
ArrowType.ArrowTypeVisitor<Integer> {
Review Comment:
We can compute buffer lengths based on the type. Obviously, this needs to be
factored out and tested and made robust and all that.
##########
java/memory/memory-core/src/main/java/org/apache/arrow/memory/BufferLedger.java:
##########
@@ -48,7 +48,7 @@ public class BufferLedger implements
ValueWithKeyIncluded<BufferAllocator>, Refe
"BufferLedger[%d]", 1) : null;
private volatile long lDestructionTime = 0;
- BufferLedger(final BufferAllocator allocator, final AllocationManager
allocationManager) {
+ public BufferLedger(final BufferAllocator allocator, final AllocationManager
allocationManager) {
Review Comment:
Ideally we'd find a way not to do this. IMO, the 'right' API might be a
formal `ForeignBufferReferenceManager` implementation that is part of the core
memory manager.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
