This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
from ef64d7a011f [feature](profile) add transaction statistics for profile
(#33488)
new 31984bb4f01 [feature](function) support quote string function #33055
new 215f402df79 [fix](nereids)when clause cannot be regarded as common sub
expression (#33358)
new 6f96e2b64af [fix](plsql) Fix handle select that fe can do without be
(#33363)
new 26d9082b9a5 [Feature](function) Add function strcmp (#33272)
new d4a67d93f3e [improve](routine-load) timely pause job if Kafka cluster
exception when consume (#33372)
new 5f30463bb3c [Chore](descriptors) remove unused codes for descriptors
(#33408)
new ab4f8fafcde [Bug](materialized-view) forbid create mv with value
column before key column (#33436)
new a4924dabb7e [enhancement](exception) enble exception logic in pipeline
execute thread (#33437)
new fe772c76e7d [fix](restore) Fix the conflict IDs between two cluster
(#33423)
new 9b7af4c0cfc [feature](schema change) unified schema change for parquet
and orc reader (#32873)
new 031f1e6a653 [case](regression) Add backup restore with NGRAM bloom
filter (#33479)
new 53336d61704 [fix](routine-load) routine load date where expression
rewrite should change over time (#33345)
new 1da1fac4ee1 [improve](load) try lock 30ms to get base_migration_lock
in rowset builder (#32243)
new fefbde89275 [log](move-memtable) improve logs in vtablet_writer_v2 and
load_stream (#33103)
new 8c66915bb56 [fix](doris compose) Fix not show ms recycler .out log in
cloud mode (#33489)
new 22c42209f72 [fix](nereids) fix a visitor bug in CommonSubExpressionOpt
(#33154)
new b035c7ceb46 [fix](catalog) fix resource is not reopen when rename
catalog (#33432)
new d2f84229ec7 [chore](test) remove some outdated datetime test case
(#33476)
new 87806a01372 [fix](debug point) fix gcc compile (#33451)
new 78b81d41505 [fix](test) remove distribute node of shape in some
regression test (#33463)
new e26a53d8a6f [fix](nereids) SemiJoinSemiJoinTransposeProject rule
didn't handle mark join correctly (#33401)
new 71f2105ac64 [fix](index change) disable light index change for cloud
mode (#33387)
new 307ad67f652 [Fix](inverted index) fix fast execute problem when need
read data opt enabled (#33526)
new b44c094bac0 [refactor](Nereids): compute unique and uniform property
respectively (#32908)
new 8884ede97a3 [fix](mtmv)add logs for mv_infos() (#33485)
new 49bb063e8c0 [fix](nereids) do not transpose semi join agg when mark
join (#32475)
new c4d61d4461f (Chore)[regression-test] fix unstable output variant case
(#33520)
The 27 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/olap/memtable.h | 2 +-
be/src/olap/rowset_builder.cpp | 4 +-
be/src/olap/schema_change.cpp | 11 +
be/src/olap/skiplist.h | 462 --------------
be/src/pipeline/exec/scan_operator.cpp | 5 +-
be/src/pipeline/exec/scan_operator.h | 8 +-
be/src/pipeline/task_scheduler.cpp | 4 +
be/src/runtime/descriptor_helper.h | 10 +-
be/src/runtime/descriptors.cpp | 132 +---
be/src/runtime/descriptors.h | 161 +----
be/src/runtime/load_stream.cpp | 26 +-
.../runtime/routine_load/data_consumer_group.cpp | 3 +-
be/src/util/debug_points.h | 8 +-
be/src/vec/data_types/data_type_struct.cpp | 9 +-
be/src/vec/exec/format/column_type_convert.cpp | 332 ++++++++++
be/src/vec/exec/format/column_type_convert.h | 537 ++++++++++++++++
be/src/vec/exec/format/format_common.h | 2 +-
be/src/vec/exec/format/orc/vorc_reader.cpp | 187 ++++--
be/src/vec/exec/format/orc/vorc_reader.h | 18 +-
be/src/vec/exec/format/parquet/decoder.cpp | 41 +-
.../exec/format/parquet/delta_bit_pack_decoder.h | 163 +++--
.../format/parquet/fix_length_dict_decoder.hpp | 125 +---
.../exec/format/parquet/fix_length_plain_decoder.h | 97 +--
.../exec/format/parquet/parquet_column_convert.cpp | 320 ++++++++--
.../exec/format/parquet/parquet_column_convert.h | 673 ++++++---------------
be/src/vec/exec/format/parquet/parquet_pred_cmp.h | 12 +-
.../exec/format/parquet/vparquet_column_reader.cpp | 38 +-
.../exec/format/parquet/vparquet_column_reader.h | 2 +-
be/src/vec/exec/scan/vscan_node.cpp | 43 +-
be/src/vec/exprs/vectorized_fn_call.cpp | 7 +-
be/src/vec/functions/function_string.cpp | 29 +
be/src/vec/functions/function_string.h | 62 ++
be/src/vec/sink/load_stream_stub.cpp | 3 +
be/src/vec/sink/writer/vtablet_writer_v2.cpp | 9 +-
be/test/olap/skiplist_test.cpp | 423 -------------
be/test/testutil/desc_tbl_builder.cpp | 7 +-
be/test/vec/exec/parquet/parquet_thrift_test.cpp | 37 +-
be/test/vec/exprs/vexpr_test.cpp | 2 +-
be/test/vec/function/function_string_test.cpp | 78 +++
.../runtime/doris-compose/resource/init_cloud.sh | 17 +-
.../sql-functions/string-functions/quote.md | 47 ++
.../sql-functions/string-functions/quote.md | 47 ++
.../apache/doris/alter/SchemaChangeHandler.java | 14 +-
.../org/apache/doris/analysis/SlotDescriptor.java | 2 +-
.../org/apache/doris/analysis/TupleDescriptor.java | 33 +-
.../doris/catalog/BuiltinScalarFunctions.java | 4 +
.../main/java/org/apache/doris/catalog/Env.java | 6 +
.../main/java/org/apache/doris/catalog/Index.java | 4 +
.../main/java/org/apache/doris/catalog/MTMV.java | 19 +
.../java/org/apache/doris/catalog/OlapTable.java | 51 +-
.../org/apache/doris/catalog/PartitionInfo.java | 30 +-
.../org/apache/doris/datasource/CatalogMgr.java | 4 +
.../org/apache/doris/load/BrokerFileGroup.java | 4 +
.../load/routineload/KafkaRoutineLoadJob.java | 8 +-
.../processor/post/CommonSubExpressionOpt.java | 33 +-
.../nereids/properties/FunctionalDependencies.java | 4 +
.../join/SemiJoinSemiJoinTransposeProject.java | 13 +-
.../rules/rewrite/TransposeSemiJoinAgg.java | 1 +
.../rules/rewrite/TransposeSemiJoinAggProject.java | 1 +
.../trees/expressions/functions/scalar/Quote.java | 69 +++
.../trees/expressions/functions/scalar/Strcmp.java | 70 +++
.../expressions/visitor/ScalarFunctionVisitor.java | 10 +
.../doris/nereids/trees/plans/AbstractPlan.java | 2 +-
.../trees/plans/BlockFuncDepsPropagation.java | 18 +-
.../nereids/trees/plans/PropagateFuncDeps.java | 20 +-
.../trees/plans/logical/LogicalAggregate.java | 108 ++--
.../trees/plans/logical/LogicalAssertNumRows.java | 33 +-
.../plans/logical/LogicalCatalogRelation.java | 23 +-
.../plans/logical/LogicalDeferMaterializeTopN.java | 29 +-
.../nereids/trees/plans/logical/LogicalExcept.java | 59 +-
.../nereids/trees/plans/logical/LogicalFilter.java | 25 +-
.../trees/plans/logical/LogicalGenerate.java | 20 +-
.../nereids/trees/plans/logical/LogicalHaving.java | 17 +-
.../trees/plans/logical/LogicalIntersect.java | 27 +-
.../nereids/trees/plans/logical/LogicalJoin.java | 130 ++--
.../nereids/trees/plans/logical/LogicalLimit.java | 30 +-
.../trees/plans/logical/LogicalOneRowRelation.java | 21 +-
.../nereids/trees/plans/logical/LogicalPlan.java | 17 +-
.../trees/plans/logical/LogicalProject.java | 65 +-
.../nereids/trees/plans/logical/LogicalRepeat.java | 19 +-
.../trees/plans/logical/LogicalSubQueryAlias.java | 26 +-
.../nereids/trees/plans/logical/LogicalTopN.java | 30 +-
.../nereids/trees/plans/logical/LogicalUnion.java | 22 +-
.../nereids/trees/plans/logical/LogicalView.java | 20 +-
.../nereids/trees/plans/logical/LogicalWindow.java | 73 ++-
.../trees/plans/physical/PhysicalProject.java | 8 +-
.../apache/doris/nereids/util/ExpressionUtils.java | 4 +-
.../apache/doris/planner/StreamLoadPlanner.java | 10 +-
.../src/main/java/org/apache/doris/plsql/Stmt.java | 2 +-
.../doris/plsql/executor/PlsqlQueryExecutor.java | 1 +
.../apache/doris/plsql/executor/PlsqlResult.java | 3 +-
.../apache/doris/plsql/executor/QueryResult.java | 4 +
.../java/org/apache/doris/qe/ConnectProcessor.java | 4 +
.../java/org/apache/doris/qe/StmtExecutor.java | 7 +
.../doris/tablefunction/MetadataGenerator.java | 37 ++
.../tablefunction/MvInfosTableValuedFunction.java | 16 +
.../properties/FunctionalDependenciesTest.java | 4 +-
.../doris/nereids/properties/UniformTest.java | 207 +++++++
...tionalDependenciesTest.java => UniqueTest.java} | 376 ++++++------
.../join/SemiJoinSemiJoinTransposeProjectTest.java | 24 +
.../doris/planner/TableFunctionPlanTest.java | 16 +-
gensrc/proto/descriptors.proto | 8 +-
gensrc/script/doris_builtins_functions.py | 7 +-
gensrc/thrift/Descriptors.thrift | 8 +-
.../datetimev1/test_datetimev1_calc.out | 18 -
.../datatype_p0/datetimev2/test_invalid_hour.out | 1 -
.../data/datatype_p0/datetimev2/test_timezone.out | 3 -
.../jdbc/test_mysql_jdbc_catalog.out | 6 +
.../hive/test_hive_schema_change.out | 52 +-
.../test_index_no_need_read_data.out | 9 +
.../filter_push_down/push_filter_through.out | 546 +++++++----------
.../transposeJoin/transposeSemiJoinAgg.out | 8 +
.../rf_prune/query10.out | 12 +-
.../nereids_tpcds_shape_sf100_p0/shape/query10.out | 12 +-
.../data/plsql_p0/test_plsql_variable.out | 6 +
regression-test/data/tpch_sf0.1_p1/sql/cse.out | 10 +-
.../data/variant_p0/compaction_sparse_column.out | 4 +-
.../test_backup_restore_ngram_bloom_filter.groovy | 168 +++++
.../datetimev1/test_datetimev1_calc.groovy | 10 +-
.../datetimev2/test_invalid_hour.groovy | 10 +-
.../datatype_p0/datetimev2/test_timezone.groovy | 5 +-
.../decimalv3/test_decimal256_index.groovy | 8 +-
.../jdbc/test_mysql_jdbc_catalog.groovy | 20 +-
.../paimon/test_paimon_catalog.groovy | 6 +-
.../hive/test_hive_schema_change.groovy | 80 ++-
.../test_build_index_fault.groovy | 3 +
.../index_change/test_index_change_2.groovy | 10 +-
.../index_change/test_index_change_3.groovy | 12 +-
.../index_change/test_index_change_4.groovy | 12 +-
.../index_change/test_index_change_6.groovy | 16 +-
.../test_index_change_on_new_column.groovy | 6 +-
.../test_index_change_with_compaction.groovy | 8 +-
.../index_change/test_pk_uk_index_change.groovy | 11 +-
.../test_add_drop_index_ignore_case_column.groovy | 12 +-
.../test_add_drop_index_with_data.groovy | 18 +-
.../test_add_drop_index_with_delete_data.groovy | 12 +-
.../inverted_index_p0/test_array_index.groovy | 4 +-
.../inverted_index_p0/test_build_index.groovy | 3 +
.../test_index_match_phrase.groovy | 30 +-
.../test_index_match_select.groovy | 30 +-
.../test_index_no_need_read_data.groovy | 274 +++++++++
.../test_index_range_in_select.groovy | 30 +-
.../test_index_range_not_in_select.groovy | 30 +-
.../suites/inverted_index_p0/test_show_data.groovy | 33 +-
.../routine_load/data/test_where_condition.csv | 20 +
.../test_routine_load_condition.groovy | 182 ++++++
regression-test/suites/mv_p0/unique/unique.groovy | 6 +
.../filter_push_down/push_filter_through.groovy | 15 +-
.../transposeJoin/transposeSemiJoinAgg.groovy | 16 +
.../suites/plsql_p0/test_plsql_variable.groovy | 27 +
.../suites/tpch_sf0.1_p1/sql/cse.groovy | 7 +-
.../variant_p0/compaction_sparse_column.groovy | 4 +-
152 files changed, 4678 insertions(+), 3310 deletions(-)
delete mode 100644 be/src/olap/skiplist.h
create mode 100644 be/src/vec/exec/format/column_type_convert.cpp
create mode 100644 be/src/vec/exec/format/column_type_convert.h
delete mode 100644 be/test/olap/skiplist_test.cpp
create mode 100644
docs/en/docs/sql-manual/sql-functions/string-functions/quote.md
create mode 100644
docs/zh-CN/docs/sql-manual/sql-functions/string-functions/quote.md
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/Quote.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/Strcmp.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/properties/UniformTest.java
copy
fe/fe-core/src/test/java/org/apache/doris/nereids/properties/{FunctionalDependenciesTest.java
=> UniqueTest.java} (50%)
create mode 100644
regression-test/suites/backup_restore/test_backup_restore_ngram_bloom_filter.groovy
create mode 100644
regression-test/suites/load_p0/routine_load/data/test_where_condition.csv
create mode 100644
regression-test/suites/load_p0/routine_load/test_routine_load_condition.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
