xgbtxy commented on code in PR #54725:
URL: https://github.com/apache/arrow/pull/54725#discussion_r4350084468
##########
cpp/src/arrow/compute/kernels/vector_selection.cc:
##########
@@ -126,15 +167,25 @@ Result<Datum> DropNullRecordBatch(const
std::shared_ptr<RecordBatch>& batch,
bit_util::SetBitsTo(dst->mutable_data(), 0, batch->num_rows(), false);
break;
}
- if (column->null_bitmap_data()) {
+ if (is_union(column->type_id())) {
+ ARROW_ASSIGN_OR_RAISE(auto validity, MakeDropNullFilter(*column, ctx));
+ ::arrow::internal::BitmapAnd(validity->values()->data(),
validity->offset(),
+ dst->data(), 0, column->length(), 0,
+ dst->mutable_data());
+ } else if (column->null_bitmap_data()) {
::arrow::internal::BitmapAnd(column->null_bitmap_data(),
column->offset(),
dst->data(), 0, column->length(), 0,
dst->mutable_data());
}
}
auto drop_null_filter = std::make_shared<BooleanArray>(batch->num_rows(),
dst);
if (drop_null_filter->true_count() == 0) {
- return RecordBatch::MakeEmpty(batch->schema(), ctx->memory_pool());
+ ArrayVector empty_columns;
+ for (const auto& column : batch->columns()) {
+ ARROW_ASSIGN_OR_RAISE(auto empty, MakeDropNullEmptyArray(column->type(),
ctx));
Review Comment:
Confirmed on `23fd210`: both new RecordBatch/Table regressions fail with
`MakeBuilder: cannot construct builder for type extension<arrow.bool8>`.
Pushed
[295917f](https://github.com/xgbtxy/arrow/commit/295917f9dcd75955753395859963732844a9bc27).
Empty outputs now use the null-array factory for extension-bearing sibling
columns as well as unions. This constructs fresh empty buffers without
requiring the nested extension builder. Ordinary types and the existing REE
construction path remain unchanged.
The regressions pair an all-null dense/sparse union with a valid
`list<bool8>` column containing `[[], []]`, and check that dropping all rows
preserves the schema and list/extension types for both RecordBatch and Table.
Both new tests pass; the complete vector-selection suite passes (173 tests, one
pre-existing disabled test). Formatting and `git diff --check` also pass. This
revision and validation were assisted by Codex.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
