This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
from 2b934dc57d [fix](Nereids) set operation syntax is not compatible with
legacy planner (#23668) (#23820)
new 3d21e1d74e [feature](TVF) support path partition keys for external
file TVF (#21648)
new 18d67a00c7 [Fix](Full compaction) Fix full compaction by table id
regressison test #23496
new dbcf903797 [fix](chore) fix env.sh build on macOS (#23676)
new 08719f9757 [improvement](profile)Add workload group in audit log and
profile (#23761)
new 72bb64ff44 [fix](json)Fix the bug that read json file Out of bounds
access (#23411)
new ffc6558fbd [Fix] spark load not found file #23502
new 4efc42407c [fix](map_agg) lost scale information for decimal type
(#23776)
new 77e96c3fb5 [fix](be) doris_be compile failed(#20932) (#20932)
new c43c95e622 [Feature](Multi-Catalog) support query doris bitmap column
in external jdbc catalog (#23021)
new 3634832bfd [Enhancement](binlog) Add ModifyPartition,
BatchModifyPartitions && ReplacePartitionOperationLog support (#23773)
new 8c340964d5 [improvement](colocate table) forbit change colocate
table's replica allocation (#23064)
new 8e84bb6071 [Fix](thrift) Add fe master check in some thrift calls
(#23757)
new f415adc434 [coverage][test] modify run-be-ut.sh for run be cov, no
need merge (#23755)
new b136846c0d [Fix](IndexColumnWriter) Add logic for
IndexedColumnWriter::add when the current page is full (#23766)
new d55e41b656 [test](ColdHotSeparation) refresh case (#23741)
new f4cd21b11d [chore](json reader) add original data to error messge for
tracing (#22803)
new 325eed6e89 [Feature](CCR) Support MoW for CCR (#22798)
new 66e1a16816 [Enhance](ip)optimize priority_ network matching logic
(#23784)
new f8838260d6 [typo](docs) modify standard-deployment doc (#23473)
new 26cfd931ee [regression-test](fix) fix case bug when define local
variable (#23785)
new 3a1bbc61bc [fix](sec) update mysql:mysql-connector-java 8.0.15 to
8.0.28 (#22776)
new 3c14cc3261 [enhancement](regression) Add a case to check if
light_schema_change=true when creating table (#18889)
new 9753f0a07d [improvement] delete duplicate code in sql_scanner.flex
(#19835)
new 458b48e22e [enhancement](jdbc catalog ) support postgresql partition
table (#23744)
new e911ccc4af [fix](load) fix cancel load failed because Job could not be
cancelled when job is finished or cancelled (#17730)
new 8ed610fd75 [fix](hive external) support set hive version when create
hive external table (#20622)
new 3248f41925 [improve](spark-load) set status fail message when spark
load job status is cancelled. (#20798)
new be40d4d64a [Test](Framework) add enable cache data args for test
CommandLine (#21852)
new 2bc34538cf [enhancement](load) support dry_run_query for load (#23680)
new 846a7faa09 [improvement](show backends) show backends print trash used
(#23792)
new 937952b554 [opt](Nereids) use avl tree to construct continuous union
operand (#23763)
new 6109a2aaf0 [fix](auth)fix after setting the user password to expire,
changing the password again will not take effect (#23426)
new e61dad08fe [fix](regressionTest) fix test_external_catalog_es p2
regression test #23838
new ec9c39bb3f [fix](load) add lock for serialize profile (#23856)
new 4d66e977a9 [fix](RemoteUDF) fix string type do not set PGenericType
#23832
new 96aec9c135 [Improve](be)check swap (#18891)
new 1a92231222 [test](regression) Reduce dependency of other database in
`test_alter_table_column` (#22547)
new 741c63df2b [fix](inverted)fix inverted index report be hostname error
#23872
new 7a4b2f4313 [Fix](bitmap index) like predicate does not work in bitmap
index (#23819)
new 3f06b50a65 [improvemnt](script) support custom lib dir to save custom
libs (#23887)
new 9a9e56bd90 [fix](spark load) not setting the file format cause null
pointer exception (#16202)
The 41 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/CMakeLists.txt | 1 +
be/src/agent/task_worker_pool.cpp | 7 +-
be/src/agent/task_worker_pool.h | 3 +-
be/src/geo/CMakeLists.txt | 4 +-
be/src/olap/data_dir.cpp | 11 +
be/src/olap/data_dir.h | 5 +
be/src/olap/olap_common.h | 1 +
be/src/olap/push_handler.cpp | 17 +-
.../rowset/segment_v2/indexed_column_writer.cpp | 9 +
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 10 +-
be/src/olap/storage_engine.cpp | 11 +-
be/src/olap/storage_engine.h | 2 +
be/src/runtime/descriptors.h | 9 +
be/src/runtime/load_channel.cpp | 3 +-
be/src/runtime/load_channel.h | 1 +
be/src/service/backend_service.cpp | 68 ++++-
be/src/util/bitmap_value.h | 98 +++++--
.../aggregate_functions/aggregate_function_map.h | 11 +-
be/src/vec/data_types/data_type_bitmap.cpp | 12 +
be/src/vec/data_types/data_type_bitmap.h | 3 +-
.../data_types/serde/data_type_string_serde.cpp | 2 +
be/src/vec/exec/format/json/new_json_reader.cpp | 21 +-
be/src/vec/exec/format/json/new_json_reader.h | 1 +
be/src/vec/exec/vjdbc_connector.cpp | 82 +++++-
be/src/vec/exec/vjdbc_connector.h | 7 +
be/src/vec/functions/function_cast.h | 24 ++
be/src/vec/sink/vtablet_sink.cpp | 4 +
bin/start_be.sh | 12 +
bin/start_fe.sh | 7 +
.../json_format_test/nest_json.json | 10 +-
.../postgresql/init/02-create-table.sql | 52 ++++
.../docker-compose/postgresql/init/04-insert.sql | 14 +
docs/en/docs/advanced/variables.md | 2 +-
.../ecosystem/udf/remote-user-defined-function.md | 49 +++-
docs/en/docs/install/standard-deployment.md | 2 +-
docs/en/docs/lakehouse/multi-catalog/faq.md | 4 +
docs/en/docs/lakehouse/multi-catalog/jdbc.md | 5 +-
docs/en/docs/lakehouse/multi-catalog/paimon.md | 5 +-
.../sql-functions/table-functions/hdfs.md | 4 +
.../sql-manual/sql-functions/table-functions/s3.md | 4 +
docs/zh-CN/docs/advanced/variables.md | 2 +-
.../ecosystem/udf/remote-user-defined-function.md | 50 +++-
docs/zh-CN/docs/install/standard-deployment.md | 2 +-
docs/zh-CN/docs/lakehouse/multi-catalog/faq.md | 4 +
docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md | 5 +-
docs/zh-CN/docs/lakehouse/multi-catalog/paimon.md | 7 +-
.../sql-functions/table-functions/hdfs.md | 5 +-
.../sql-manual/sql-functions/table-functions/s3.md | 3 +
env.sh | 4 +-
.../java/org/apache/doris/jdbc/JdbcExecutor.java | 50 ++++
.../main/java/org/apache/doris/alter/Alter.java | 3 +
.../apache/doris/analysis/FunctionCallExpr.java | 4 +
.../org/apache/doris/analysis/S3TvfLoadStmt.java | 6 +
.../org/apache/doris/binlog/BinlogManager.java | 26 ++
.../java/org/apache/doris/catalog/DiskInfo.java | 16 +-
.../main/java/org/apache/doris/catalog/Env.java | 13 +-
.../java/org/apache/doris/catalog/HiveTable.java | 6 +
.../java/org/apache/doris/catalog/OlapTable.java | 6 +
.../apache/doris/common/proc/BackendsProcDir.java | 10 +-
.../doris/common/profile/SummaryProfile.java | 10 +-
.../apache/doris/datasource/InternalCatalog.java | 3 +
.../datasource/jdbc/client/JdbcMySQLClient.java | 4 +
.../jdbc/client/JdbcPostgreSQLClient.java | 2 +-
.../java/org/apache/doris/load/loadv2/LoadJob.java | 2 +-
.../org/apache/doris/load/loadv2/LoadManager.java | 7 +-
.../doris/load/loadv2/SparkEtlJobHandler.java | 2 +
.../doris/mysql/privilege/PasswordPolicy.java | 8 +
.../mysql/privilege/PasswordPolicyManager.java | 7 +
.../doris/nereids/parser/LogicalPlanBuilder.java | 84 ++++--
.../doris/persist/BatchModifyPartitionsInfo.java | 34 +++
.../java/org/apache/doris/persist/EditLog.java | 17 +-
.../persist/ReplacePartitionOperationLog.java | 21 +-
.../apache/doris/planner/SingleNodePlanner.java | 2 +-
.../apache/doris/planner/external/TVFScanNode.java | 2 +-
.../java/org/apache/doris/plugin/AuditEvent.java | 8 +
.../java/org/apache/doris/qe/ConnectContext.java | 11 +
.../java/org/apache/doris/qe/ConnectProcessor.java | 1 +
.../java/org/apache/doris/qe/StmtExecutor.java | 3 +
.../resource/workloadgroup/WorkloadGroupMgr.java | 1 +
.../org/apache/doris/service/FrontendOptions.java | 47 ++--
.../apache/doris/service/FrontendServiceImpl.java | 45 +++
.../main/java/org/apache/doris/system/Backend.java | 13 +
.../tablefunction/BackendsTableValuedFunction.java | 1 +
.../ExternalFileTableValuedFunction.java | 37 ++-
.../tablefunction/HdfsTableValuedFunction.java | 9 +-
.../doris/tablefunction/MetadataGenerator.java | 3 +
.../doris/tablefunction/S3TableValuedFunction.java | 3 +-
fe/fe-core/src/main/jflex/sql_scanner.flex | 7 -
.../java/org/apache/doris/alter/AlterTest.java | 61 +++++
.../org/apache/doris/catalog/HiveTableTest.java | 8 +
.../doris/clone/TabletRepairAndBalanceTest.java | 5 +-
.../doris/utframe/DemoMultiBackendsTest.java | 2 +-
.../org/apache/doris/load/loadv2/dpp/SparkDpp.java | 4 +-
gensrc/thrift/FrontendService.thrift | 2 +
gensrc/thrift/MasterService.thrift | 1 +
gensrc/thrift/Status.thrift | 2 +
.../test_full_compaction_by_table_id.out | 2 +-
.../jdbc/test_doris_jdbc_catalog_query_bitmap.out | 51 ++++
.../jdbc/test_pg_jdbc_catalog.out | 40 +++
.../data/external_table_p0/tvf/test_hdfs_tvf.out | 14 +-
.../inverted_index_p0/test_index_like_select.out | 16 ++
.../data/query_p0/aggregate/map_agg.out | 9 +-
.../insert => schema_change_p0}/baseall.txt | 0
.../org/apache/doris/regression/Config.groovy | 1 +
.../apache/doris/regression/ConfigOptions.groovy | 10 +-
.../pipeline/p0/conf/regression-conf.groovy | 2 +-
.../suites/account_p0/test_alter_user.groovy | 34 ++-
.../test_binlog_config_change.groovy | 108 ++++++++
.../test_create_table_with_binlog_config.groovy | 88 ++++++
.../ccr_mow_syncer_p0/test_get_binlog.groovy | 142 ++++++++++
.../ccr_mow_syncer_p0/test_ingest_binlog.groovy | 121 +++++++++
.../ccr_mow_syncer_p0/test_multi_buckets.groovy | 98 +++++++
.../suites/ccr_mow_syncer_p0/test_txn_case.groovy | 301 +++++++++++++++++++++
.../ccr_mow_syncer_p1/test_backup_restore.groovy | 68 +++++
.../modify_partition_add_policy.groovy | 39 +++
.../create_table_use_partition_policy.groovy | 10 +-
.../table_modify_resouce_and_policy.groovy | 6 +-
.../test_full_compaction_by_table_id.groovy | 2 +-
.../csv_header_p0/test_csv_with_header.groovy | 6 +-
.../suites/export/test_array_export.groovy | 2 +-
.../suites/export/test_map_export.groovy | 2 +-
.../test_doris_jdbc_catalog_query_bitmap.groovy | 124 +++++++++
.../jdbc/test_pg_jdbc_catalog.groovy | 10 +
.../external_table_p0/tvf/test_backends_tvf.groovy | 2 +-
.../es/test_external_catalog_es.groovy | 11 +-
.../test_index_like_select.groovy | 56 ++++
.../broker_load/test_tvf_based_broker_load.groovy | 105 ++++++-
.../mv_p0/test_dup_mv_year/test_dup_mv_year.groovy | 2 +-
.../nereids_syntax_p0/information_schema.groovy | 2 +-
.../suites/query_p0/aggregate/map_agg.groovy | 58 +++-
.../test_alter_colocate_table.groovy | 132 +++++++++
.../test_alter_table_column.groovy | 30 +-
.../test_light_schema_change_default.groovy} | 30 +-
run-be-ut.sh | 2 +-
.../org/apache/doris/udf/FunctionServiceImpl.java | 39 ++-
samples/doris-demo/spring-jdbc-demo/pom.xml | 2 +-
136 files changed, 2841 insertions(+), 234 deletions(-)
create mode 100644
regression-test/data/external_table_p0/jdbc/test_doris_jdbc_catalog_query_bitmap.out
copy regression-test/data/{load_p0/insert => schema_change_p0}/baseall.txt
(100%)
create mode 100644
regression-test/suites/ccr_mow_syncer_p0/test_binlog_config_change.groovy
create mode 100644
regression-test/suites/ccr_mow_syncer_p0/test_create_table_with_binlog_config.groovy
create mode 100644
regression-test/suites/ccr_mow_syncer_p0/test_get_binlog.groovy
create mode 100644
regression-test/suites/ccr_mow_syncer_p0/test_ingest_binlog.groovy
create mode 100644
regression-test/suites/ccr_mow_syncer_p0/test_multi_buckets.groovy
create mode 100644
regression-test/suites/ccr_mow_syncer_p0/test_txn_case.groovy
create mode 100644
regression-test/suites/ccr_mow_syncer_p1/test_backup_restore.groovy
create mode 100644
regression-test/suites/external_table_p0/jdbc/test_doris_jdbc_catalog_query_bitmap.groovy
create mode 100644
regression-test/suites/schema_change_p0/test_alter_colocate_table.groovy
copy regression-test/suites/{nereids_syntax_p0/information_schema.groovy =>
schema_change_p0/test_light_schema_change_default.groovy} (54%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
