aokolnychyi commented on a change in pull request #3533:
URL: https://github.com/apache/iceberg/pull/3533#discussion_r749653784
##########
File path:
arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedArrowReader.java
##########
@@ -407,35 +412,55 @@ public void setBatchSize(int batchSize) {
}
private static final class PositionVectorReader extends
VectorizedArrowReader {
+ private final Field arrowField =
ArrowSchemaUtil.convert(MetadataColumns.ROW_POSITION);
+ private final BufferAllocator bufferAllocator =
ArrowAllocation.rootAllocator();
+ private final boolean setArrowValidityVector;
private long rowStart;
+ private int batchSize;
+ private FieldVector vec;
private NullabilityHolder nulls;
+ PositionVectorReader(boolean setArrowValidityVector) {
+ this.setArrowValidityVector = setArrowValidityVector;
+ }
+
@Override
public VectorHolder read(VectorHolder reuse, int numValsToRead) {
- Field arrowField = ArrowSchemaUtil.convert(MetadataColumns.ROW_POSITION);
- FieldVector vec =
arrowField.createVector(ArrowAllocation.rootAllocator());
-
- if (reuse != null) {
- vec.setValueCount(0);
- nulls.reset();
+ if (reuse == null) {
+ this.vec = newVector();
+ this.nulls = newNullabilityHolder();
} else {
- ((BigIntVector) vec).allocateNew(numValsToRead);
- for (int i = 0; i < numValsToRead; i += 1) {
- vec.getDataBuffer().setLong(i * Long.BYTES, rowStart + i);
- }
- for (int i = 0; i < numValsToRead; i += 1) {
- BitVectorHelper.setBit(vec.getValidityBuffer(), i);
+ vec.setValueCount(0);
Review comment:
While I agree with how the reuse logic should behave theoretically, it
is not we have in `VectorizedArrowReader`. I looked at that part as an example
of how it should behave. I am a bit worried about unforeseen consequences of
changing just this place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
