Gabriel39 commented on code in PR #67441:
URL: https://github.com/apache/doris/pull/67441#discussion_r3912509796
##########
be/src/exprs/function/array/function_array_index.h:
##########
@@ -218,6 +220,21 @@ class FunctionArrayIndex : public IFunction {
return _execute_dispatch(block, arguments, result, input_rows_count);
}
+ ZoneMapFilterResult evaluate_zonemap_filter(const ZoneMapEvalContext& ctx,
+ const VExprSPtrs& arguments)
const override {
+ if constexpr (!std::is_same_v<ConcreteAction, ArrayContainsAction>) {
+ return unsupported_zonemap_filter(ctx);
+ }
+ return expr_zonemap::evaluate_array_contains_zonemap(ctx, arguments);
Review Comment:
Fixed in 6b293ea164. `evaluate_array_contains_zonemap()` now returns
unsupported when the caller binds the logical ARRAY instead of the repeated
element type, avoiding the invariant failure. The regression test covers the
logical ARRAY context shared by Segment and legacy Parquet.
##########
be/test/format_v2/parquet/parquet_statistics_test.cpp:
##########
@@ -1820,6 +1853,105 @@ TEST(NativeParquetStatisticsTest,
TypeDefinedBoundsRequireSupportedColumnOrder)
EXPECT_TRUE(selected_ranges.empty());
}
+TEST(NativeParquetStatisticsTest,
ArrayContainsUsesRepeatedLeafFooterStatisticsOnly) {
+ const auto encode_int32 = [](int32_t value) {
+ std::string bytes(sizeof(value), '\0');
+ memcpy(bytes.data(), &value, sizeof(value));
+ return bytes;
+ };
+
+ const auto leaf_type = std::make_shared<DataTypeInt32>();
+ const auto array_type = std::make_shared<DataTypeArray>(leaf_type);
+ auto root_schema =
std::make_unique<format::parquet::ParquetColumnSchema>();
+ root_schema->kind = format::parquet::ParquetColumnSchemaKind::LIST;
+ root_schema->local_id = 0;
+ root_schema->name = "items";
+ root_schema->type = array_type;
+ root_schema->max_repetition_level = 1;
+ auto leaf_schema =
std::make_unique<format::parquet::ParquetColumnSchema>();
+ leaf_schema->kind = format::parquet::ParquetColumnSchemaKind::PRIMITIVE;
+ leaf_schema->local_id = 0;
+ leaf_schema->name = "element";
+ leaf_schema->leaf_column_id = 0;
+ leaf_schema->type = leaf_type;
+ leaf_schema->type_descriptor.doris_type = leaf_type;
+ leaf_schema->type_descriptor.physical_type = tparquet::Type::INT32;
+ leaf_schema->max_repetition_level = 1;
+ root_schema->children.push_back(std::move(leaf_schema));
+ std::vector<std::unique_ptr<format::parquet::ParquetColumnSchema>> schema;
+ schema.push_back(std::move(root_schema));
+
+ tparquet::Statistics statistics;
+ statistics.__set_min_value(encode_int32(10));
+ statistics.__set_max_value(encode_int32(20));
+ statistics.__set_null_count(0);
+ tparquet::ColumnMetaData column_metadata;
+ column_metadata.__set_type(tparquet::Type::INT32);
+ column_metadata.__set_num_values(2);
+ column_metadata.__set_statistics(statistics);
+ tparquet::ColumnChunk chunk;
+ chunk.__set_meta_data(column_metadata);
+ tparquet::RowGroup row_group;
+ row_group.__set_columns({chunk});
+ row_group.__set_num_rows(1);
+ tparquet::ColumnOrder order;
+ order.__set_TYPE_ORDER(tparquet::TypeDefinedOrder());
+ tparquet::FileMetaData metadata;
+ metadata.__set_column_orders({order});
+ metadata.__set_row_groups({row_group});
+
+ const auto select_for_value = [&](int32_t value,
+ format::parquet::ParquetPruningStats*
pruning_stats) {
+ format::FileScanRequest request;
Review Comment:
Fixed in 6b293ea164. `array_contains` is now admitted as a total predicate
for selected-row execution, and a TableReader integration test verifies that a
prepared production expression reaches the metadata-safe request prefix. The
Page Index gate now requires an actually page-evaluable, non-repeated probe,
with a regression assertion that an array-only request does not trigger Page
Index I/O.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
