This is an automated email from the ASF dual-hosted git repository.
lihaopeng pushed a change to branch topn-lazy-materialize-poc
in repository https://gitbox.apache.org/repos/asf/doris.git
discard df601ffe210 fe
discard d0b93cc5e7d Support topn-lazy-materialize
add 9ce55d1a0de [Enhancement] (nereids) implement cleanQueryStatsCommand
in nereids (#49120)
add 0056b11b2f8 [Bug](function) fix date_floor default value not
0001-01-01 00:00:00 (#50479)
add 30b7f20f1f1 [improve](function) support regexp_replace function with
ignore flag options (#50245)
add 93ffdbf8659 [Enhancement](audit) Add fields related to file cache hits
in the audit log table (#48041)
add 77d75e24418 [test](case) add some test case about partition sort hash
types (#50318)
add 559b1837a3f [refactor](type) Use DataType to replace TypeDescriptor
(#50290)
add 909a95d6b39 [fix](geo) Fix ST_Contains behavior (#50115)
add a68c8df37fb [Bug](agg-state) fix agg state inner type not conversioned
(#50418)
add 75d68f6ad64 [fix](core) Fix core caused by
VDataStreamMgr::transmit_block (#50560)
add 94ede109c21 [opt](cloud) Add config to control sync_rowsets
parallelism when init scanners (#49420)
add 1bef8bfc7a5 Support topn-lazy-materialize
add 17283e04319 fe
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (df601ffe210)
\
N -- N -- N refs/heads/topn-lazy-materialize-poc (17283e04319)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/cloud/config.cpp | 1 +
be/src/cloud/config.h | 2 +
be/src/exec/es/es_scroll_parser.cpp | 10 +-
be/src/exec/odbc_connector.cpp | 7 +-
be/src/exec/rowid_fetcher.cpp | 40 +-
be/src/exec/schema_scanner.cpp | 4 +-
.../schema_active_queries_scanner.cpp | 4 +-
.../schema_scanner/schema_backend_active_tasks.cpp | 5 +-
.../schema_backend_kerberos_ticket_cache.cpp | 5 +-
.../schema_catalog_meta_cache_stats_scanner.cpp | 4 +-
.../schema_file_cache_statistics.cpp | 5 +-
.../schema_scanner/schema_partitions_scanner.cpp | 4 +-
.../exec/schema_scanner/schema_routine_scanner.cpp | 4 +-
.../schema_table_options_scanner.cpp | 4 +-
.../schema_table_properties_scanner.cpp | 4 +-
.../schema_workload_group_privileges.cpp | 4 +-
...chema_workload_group_resource_usage_scanner.cpp | 5 +-
.../schema_workload_groups_scanner.cpp | 4 +-
.../schema_workload_sched_policy_scanner.cpp | 4 +-
be/src/exec/table_connector.cpp | 16 +-
be/src/exec/table_connector.h | 5 +-
be/src/exec/tablet_info.cpp | 10 +-
be/src/exec/tablet_info.h | 3 +-
be/src/exprs/string_functions.cpp | 16 +-
be/src/exprs/string_functions.h | 3 +-
be/src/geo/geo_types.cpp | 43 +-
be/src/olap/iterators.h | 2 +-
be/src/olap/match_predicate.cpp | 31 +-
be/src/olap/push_handler.cpp | 18 +-
be/src/olap/push_handler.h | 2 +-
be/src/olap/rowset/rowset_reader_context.h | 2 +-
be/src/olap/rowset/segment_v2/column_reader.cpp | 2 +-
be/src/olap/rowset/segment_v2/column_writer.cpp | 2 +-
be/src/olap/rowset/segment_v2/segment.h | 5 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 2 +-
be/src/olap/tablet_reader.cpp | 4 +-
be/src/olap/tablet_reader.h | 2 +-
be/src/olap/tablet_schema.cpp | 2 +-
be/src/pipeline/common/partition_sort_utils.cpp | 5 -
be/src/pipeline/common/partition_sort_utils.h | 9 +-
be/src/pipeline/exec/analytic_sink_operator.cpp | 2 +-
be/src/pipeline/exec/hashjoin_build_sink.cpp | 3 +-
be/src/pipeline/exec/hashjoin_probe_operator.cpp | 2 +-
be/src/pipeline/exec/olap_scan_operator.cpp | 5 +-
.../pipeline/exec/partition_sort_sink_operator.h | 7 +-
be/src/pipeline/exec/scan_operator.cpp | 70 +--
be/src/pipeline/exec/scan_operator.h | 5 +-
be/src/pipeline/exec/schema_scan_operator.cpp | 15 +-
be/src/runtime/descriptors.cpp | 41 +-
be/src/runtime/descriptors.h | 12 +-
be/src/runtime/fold_constant_executor.cpp | 32 +-
be/src/runtime/fold_constant_executor.h | 2 +-
be/src/runtime/fragment_mgr.cpp | 2 +-
be/src/runtime/primitive_type.h | 8 +
be/src/runtime/types.cpp | 329 --------------
be/src/runtime/types.h | 239 -----------
be/src/runtime_filter/runtime_filter.cpp | 8 +-
be/src/runtime_filter/runtime_filter_consumer.cpp | 22 +-
be/src/runtime_filter/runtime_filter_merger.h | 4 +-
be/src/runtime_filter/utils.cpp | 18 +-
be/src/runtime_filter/utils.h | 6 +-
be/src/service/internal_service.cpp | 5 +-
be/src/udf/udf.cpp | 11 +-
be/src/udf/udf.h | 17 +-
be/src/util/arrow/row_batch.cpp | 48 ++-
be/src/util/arrow/row_batch.h | 4 +-
be/src/vec/columns/column_nullable.cpp | 4 +-
be/src/vec/core/types.h | 10 +
be/src/vec/data_types/data_type.h | 44 +-
be/src/vec/data_types/data_type_agg_state.h | 6 +-
be/src/vec/data_types/data_type_array.h | 21 +-
be/src/vec/data_types/data_type_bitmap.h | 5 +-
be/src/vec/data_types/data_type_date.h | 4 +-
be/src/vec/data_types/data_type_date_time.h | 4 +-
be/src/vec/data_types/data_type_decimal.cpp | 53 +++
be/src/vec/data_types/data_type_decimal.h | 33 +-
be/src/vec/data_types/data_type_factory.cpp | 473 +++++++++++++++------
be/src/vec/data_types/data_type_factory.hpp | 23 +-
.../vec/data_types/data_type_fixed_length_object.h | 5 +-
be/src/vec/data_types/data_type_hll.h | 2 +-
be/src/vec/data_types/data_type_ipv4.h | 2 +-
be/src/vec/data_types/data_type_ipv6.h | 4 +-
be/src/vec/data_types/data_type_jsonb.h | 4 +-
be/src/vec/data_types/data_type_map.h | 24 +-
be/src/vec/data_types/data_type_nothing.h | 6 +-
be/src/vec/data_types/data_type_nullable.h | 15 +-
be/src/vec/data_types/data_type_number_base.h | 20 +-
be/src/vec/data_types/data_type_object.h | 7 +-
be/src/vec/data_types/data_type_quantilestate.h | 4 +-
be/src/vec/data_types/data_type_string.h | 34 +-
be/src/vec/data_types/data_type_struct.h | 36 +-
be/src/vec/data_types/data_type_time.h | 4 +-
be/src/vec/data_types/data_type_time_v2.h | 11 +-
.../vec/exec/format/arrow/arrow_stream_reader.cpp | 2 +-
be/src/vec/exec/format/arrow/arrow_stream_reader.h | 2 +-
be/src/vec/exec/format/avro/avro_jni_reader.cpp | 39 +-
be/src/vec/exec/format/avro/avro_jni_reader.h | 7 +-
be/src/vec/exec/format/column_type_convert.cpp | 85 ++--
be/src/vec/exec/format/column_type_convert.h | 15 +-
be/src/vec/exec/format/csv/csv_reader.cpp | 37 +-
be/src/vec/exec/format/csv/csv_reader.h | 8 +-
be/src/vec/exec/format/generic_reader.h | 6 +-
be/src/vec/exec/format/jni_reader.cpp | 4 +-
be/src/vec/exec/format/jni_reader.h | 3 +-
be/src/vec/exec/format/json/new_json_reader.cpp | 91 ++--
be/src/vec/exec/format/json/new_json_reader.h | 9 +-
be/src/vec/exec/format/orc/vorc_reader.cpp | 103 ++---
be/src/vec/exec/format/orc/vorc_reader.h | 6 +-
.../vec/exec/format/parquet/bool_plain_decoder.cpp | 2 +-
.../vec/exec/format/parquet/bool_rle_decoder.cpp | 2 +-
.../exec/format/parquet/parquet_column_convert.cpp | 45 +-
.../exec/format/parquet/parquet_column_convert.h | 28 +-
be/src/vec/exec/format/parquet/parquet_pred_cmp.h | 4 +-
be/src/vec/exec/format/parquet/schema_desc.cpp | 186 ++++----
be/src/vec/exec/format/parquet/schema_desc.h | 19 +-
.../exec/format/parquet/vparquet_column_reader.cpp | 64 +--
.../exec/format/parquet/vparquet_group_reader.cpp | 5 +-
be/src/vec/exec/format/parquet/vparquet_reader.cpp | 18 +-
be/src/vec/exec/format/parquet/vparquet_reader.h | 5 +-
be/src/vec/exec/format/table/equality_delete.cpp | 14 +-
be/src/vec/exec/format/table/hudi_jni_reader.cpp | 2 +-
be/src/vec/exec/format/table/hudi_jni_reader.h | 3 +-
be/src/vec/exec/format/table/iceberg_reader.cpp | 10 +-
be/src/vec/exec/format/table/iceberg_reader.h | 9 +-
.../vec/exec/format/table/lakesoul_jni_reader.cpp | 2 +-
be/src/vec/exec/format/table/lakesoul_jni_reader.h | 3 +-
.../exec/format/table/max_compute_jni_reader.cpp | 7 +-
.../vec/exec/format/table/max_compute_jni_reader.h | 3 +-
be/src/vec/exec/format/table/paimon_jni_reader.cpp | 4 +-
be/src/vec/exec/format/table/paimon_jni_reader.h | 3 +-
be/src/vec/exec/format/table/table_format_reader.h | 5 +-
.../format/table/transactional_hive_reader.cpp | 4 +-
.../exec/format/table/transactional_hive_reader.h | 1 -
.../format/table/trino_connector_jni_reader.cpp | 4 +-
.../exec/format/table/trino_connector_jni_reader.h | 3 +-
be/src/vec/exec/format/wal/wal_reader.cpp | 2 +-
be/src/vec/exec/format/wal/wal_reader.h | 2 +-
be/src/vec/exec/jni_connector.cpp | 44 +-
be/src/vec/exec/jni_connector.h | 6 +-
be/src/vec/exec/scan/file_scanner.cpp | 46 +-
be/src/vec/exec/scan/file_scanner.h | 7 +-
be/src/vec/exec/scan/meta_scanner.cpp | 28 +-
be/src/vec/exec/scan/olap_scanner.cpp | 11 +-
be/src/vec/exec/vjdbc_connector.cpp | 19 +-
.../exprs/table_function/udf_table_function.cpp | 3 +-
be/src/vec/exprs/varray_literal.cpp | 2 -
be/src/vec/exprs/vectorized_agg_fn.cpp | 15 +-
be/src/vec/exprs/vectorized_agg_fn.h | 2 -
be/src/vec/exprs/vectorized_fn_call.cpp | 2 +-
be/src/vec/exprs/vexpr.cpp | 33 +-
be/src/vec/exprs/vexpr.h | 7 +-
be/src/vec/exprs/vexpr_context.cpp | 4 +-
be/src/vec/exprs/vexpr_context.h | 4 +-
be/src/vec/exprs/vinfo_func.cpp | 4 +-
be/src/vec/exprs/vmap_literal.cpp | 2 -
be/src/vec/exprs/vslot_ref.cpp | 5 +-
be/src/vec/functions/array/function_array_index.h | 6 +-
.../vec/functions/array/function_arrays_overlap.h | 3 +-
be/src/vec/functions/function_binary_arithmetic.h | 9 +-
be/src/vec/functions/function_collection_in.h | 3 +-
.../vec/functions/function_datetime_floor_ceil.cpp | 11 +-
be/src/vec/functions/function_ip.h | 2 +-
be/src/vec/functions/function_multi_match.cpp | 2 +-
be/src/vec/functions/function_regexp.cpp | 176 ++++++--
be/src/vec/functions/function_timestamp.cpp | 3 +-
be/src/vec/functions/functions_comparison.h | 2 +-
be/src/vec/functions/in.h | 16 +-
be/src/vec/functions/match.cpp | 2 +-
be/src/vec/olap/olap_data_convertor.cpp | 2 +-
be/src/vec/runtime/partitioner.cpp | 5 +-
be/src/vec/runtime/vcsv_transformer.cpp | 2 +-
be/src/vec/runtime/vdata_stream_mgr.cpp | 16 +-
be/src/vec/runtime/vdata_stream_mgr.h | 8 +-
be/src/vec/runtime/vdata_stream_recvr.cpp | 4 +-
be/src/vec/runtime/vdata_stream_recvr.h | 2 +-
be/src/vec/runtime/vdatetime_value.h | 11 +-
be/src/vec/runtime/vorc_transformer.cpp | 42 +-
be/src/vec/runtime/vorc_transformer.h | 2 +-
be/src/vec/runtime/vparquet_transformer.cpp | 2 +-
be/src/vec/runtime/vparquet_transformer.h | 4 -
be/src/vec/sink/vmysql_result_writer.cpp | 5 +-
be/src/vec/sink/vtablet_block_convertor.cpp | 116 ++---
be/src/vec/sink/vtablet_block_convertor.h | 17 +-
.../sink/writer/iceberg/partition_transformers.cpp | 48 +--
.../sink/writer/iceberg/partition_transformers.h | 307 ++++++-------
.../sink/writer/iceberg/viceberg_table_writer.cpp | 35 +-
.../sink/writer/iceberg/viceberg_table_writer.h | 9 +-
be/src/vec/sink/writer/vfile_result_writer.cpp | 13 +-
be/src/vec/sink/writer/vhive_table_writer.cpp | 24 +-
be/src/vec/sink/writer/vhive_table_writer.h | 3 +-
be/src/vec/sink/writer/vmysql_table_writer.cpp | 9 +-
be/test/exprs/string_functions_test.cpp | 13 +-
be/test/geo/geo_types_test.cpp | 2 +-
be/test/olap/delta_writer_cluster_key_test.cpp | 3 +-
be/test/olap/delta_writer_test.cpp | 10 +-
be/test/olap/memtable_memory_limiter_test.cpp | 3 +-
be/test/olap/segment_cache_test.cpp | 3 +-
be/test/olap/tablet_cooldown_test.cpp | 3 +-
be/test/pipeline/exec/data_queue_test.cpp | 8 +
be/test/pipeline/exec/vdata_stream_recvr_test.cpp | 56 ++-
.../operator/analytic_sink_operator_test.cpp | 182 ++++++++
.../operator/hashjoin_probe_operator_test.cpp | 8 +-
.../operator/partition_sort_sink_operator_test.cpp | 195 ++++++++-
be/test/pipeline/operator/repeat_operator_test.cpp | 3 -
.../operator/scan_normalize_predicate_test.cpp | 29 ++
be/test/runtime/snapshot_loader_test.cpp | 3 +-
.../runtime_filter_consumer_helper_test.cpp | 3 +
be/test/runtime_filter/utils_test.cpp | 13 +-
be/test/testutil/desc_tbl_builder.cpp | 33 +-
be/test/testutil/desc_tbl_builder.h | 10 +-
be/test/testutil/function_utils.cpp | 9 +-
be/test/testutil/function_utils.h | 5 +-
be/test/testutil/mock/mock_descriptors.h | 14 +-
be/test/vec/columns/column_string_test.cpp | 2 +-
be/test/vec/data_types/common_data_type_test.h | 9 +-
.../vec/data_types/data_type_agg_state_test.cpp | 4 +-
be/test/vec/data_types/data_type_array_test.cpp | 8 +-
be/test/vec/data_types/data_type_bitmap_test.cpp | 5 +-
be/test/vec/data_types/data_type_decimal_test.cpp | 46 +-
.../data_type_fixed_length_object_test.cpp | 4 +-
be/test/vec/data_types/data_type_hll_test.cpp | 5 +-
be/test/vec/data_types/data_type_ip_test.cpp | 10 +-
be/test/vec/data_types/data_type_number_test.cpp | 21 +-
.../data_types/data_type_quantile_state_test.cpp | 5 +-
be/test/vec/data_types/data_type_string_test.cpp | 5 +-
.../serde/data_type_serde_arrow_test.cpp | 330 +++++++-------
.../serde/data_type_serde_mysql_test.cpp | 63 ++-
be/test/vec/exec/column_type_convert_test.cpp | 168 ++++----
.../exec/format/parquet/parquet_reader_test.cpp | 6 +-
.../exec/format/parquet/parquet_thrift_test.cpp | 37 +-
be/test/vec/exec/orc_reader_test.cpp | 28 +-
be/test/vec/exprs/vexpr_test.cpp | 23 +-
.../vec/function/function_money_format_test.cpp | 11 +-
be/test/vec/function/function_test_util.cpp | 61 ++-
be/test/vec/function/function_test_util.h | 50 ++-
be/test/vec/jsonb/serialize_test.cpp | 60 +--
.../writer/iceberg/partition_transformers_test.cpp | 50 ++-
.../utils/arrow_column_to_doris_column_test.cpp | 8 +-
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 10 +-
.../doris/catalog/InternalSchemaInitializer.java | 56 ++-
.../doris/nereids/parser/LogicalPlanBuilder.java | 16 +
.../functions/executable/TimeRoundSeries.java | 7 +-
.../functions/scalar/RegexpReplace.java | 45 +-
.../functions/scalar/RegexpReplaceOne.java | 45 +-
.../apache/doris/nereids/trees/plans/PlanType.java | 1 +
.../plans/commands/CleanQueryStatsCommand.java} | 115 +++--
.../trees/plans/commands/info/TableNameInfo.java | 13 +
.../trees/plans/visitor/CommandVisitor.java | 5 +
.../apache/doris/nereids/types/AggStateType.java | 6 +
.../catalog/InternalSchemaInitializerTest.java | 228 ++++++++++
.../plans/commands/CleanQueryStatsCommandTest.java | 108 +++++
.../data/correctness_p0/test_time_round.out | Bin 5047 -> 5047 bytes
.../agg_state/decimal_sum/test_decimal_sum.out} | Bin 126 -> 130 bytes
.../data/nereids_function_p0/scalar_function/D.out | Bin 34248 -> 34248 bytes
.../data/nereids_function_p0/scalar_function/H.out | Bin 17180 -> 17180 bytes
.../data/nereids_function_p0/scalar_function/M.out | Bin 41160 -> 41160 bytes
.../data/nereids_function_p0/scalar_function/S.out | Bin 53624 -> 53624 bytes
.../data/nereids_function_p0/scalar_function/W.out | Bin 23595 -> 23595 bytes
.../data/nereids_function_p0/scalar_function/Y.out | Bin 18467 -> 18467 bytes
.../spatial_functions/test_gis_function.out | Bin 6772 -> 7926 bytes
.../data/nereids_syntax_p0/test_regexp_replace.out | Bin 266 -> 598 bytes
.../datetime_functions/test_date_floor_ceil.out | Bin 748 -> 886 bytes
.../query_p0/test_partition_sort_hash_types.out | Bin 0 -> 2361 bytes
.../agg_state/decimal_sum/test_decimal_sum.groovy} | 32 +-
.../tvf/test_hdfs_parquet_group4.groovy | 4 +-
.../tvf/test_hdfs_parquet_group5.groovy | 4 +-
.../tvf/test_hdfs_parquet_group6.groovy | 6 +-
.../spatial_functions/test_gis_function.groovy | 61 ++-
.../nereids_syntax_p0/test_regexp_replace.groovy | 20 +
.../datetime_functions/test_date_floor_ceil.groovy | 13 +-
.../query_p0/test_partition_sort_hash_types.groovy | 65 +++
271 files changed, 3896 insertions(+), 2711 deletions(-)
copy
fe/fe-core/src/main/java/org/apache/doris/{analysis/CleanQueryStatsStmt.java =>
nereids/trees/plans/commands/CleanQueryStatsCommand.java} (54%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/CleanQueryStatsCommandTest.java
copy
regression-test/data/{correctness/test_insert_table_with_dump_nereids_memo.out
=> datatype_p0/agg_state/decimal_sum/test_decimal_sum.out} (84%)
create mode 100644
regression-test/data/query_p0/test_partition_sort_hash_types.out
copy regression-test/suites/{correctness/test_set_session_var_exception.groovy
=> datatype_p0/agg_state/decimal_sum/test_decimal_sum.groovy} (64%)
create mode 100644
regression-test/suites/query_p0/test_partition_sort_hash_types.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
