This is an automated email from the ASF dual-hosted git repository.
panxiaolei pushed a change to branch dev_join
in repository https://gitbox.apache.org/repos/asf/doris.git
omit 8b6c0951c6d update
omit cfc28b3453f update
omit cf626619449 update
omit 2b2afa8d78c merge block to single block on join/set node
add cf073ec8ce6 [runtimefilter](nerieds)support Non equal runtime filter
for nested loop join #25193
add e3d0e557945 [feature-wip] (Nereids) Support transforming trino dialect
SQL to logical plan (#21855)
add dd1c4f42185 [fix](regression) fix group commit stream load regression
test (#25469)
add 7fd876f3a22 [fix](planner)should call SlotRef'smaterializeSrcExpr()
method if the slotRef is materialized (#25467)
add f9df3bae613 [Enhancement](functions) change some nullable mode and
clear some smooth upgrade (#25334)
add 72920fbd1d1 [Improvement](materialized-view) set job failed when
toAgentTaskRequest meet error (#25358)
add bfc602f3439 [compile](fix) fix ubsan compile error (#25473)
add eaf5febc970 [enhancement](cooldown) Improve cooldown logs (#25432)
add e9157a3dba6 [fix](path gc) fix data dir path gc (#25420)
add f9a80ecdab5 [improvement](sync version) fe sync version with be
(#25236)
add fe1980d7f27 [docs](docs) Add release note 2.0.2 (#25375)
add 59ebbb351ee [feature](merge-cloud) Enable write into cache when
uploading file to s3 using s3 file writer (#24364)
add f75ee49cb46 [chore](fmt) Remove stringstream by fmt (#25474)
add f1a5e393c7a [feature](insert) Support group commit insert use new
syntax like insert into table_id(xxx) (#25484)
add a194a154423 [improvement](tablet schedule) colocate balance between
all groups (#23543)
add cda8fb6b8b1 [fix](load) return Status when error in
RowsetWriter::build (#25381)
add 85b84976249 [fix](Tvf) return empty set when tvf queries an empty file
or an error uri (#25280)
add ef7d8aa99a0 [fix](be)confix bug of converting outer join probe block
to nullable (#25492)
add a364a24ac22 [Enhance](regression) add hive out file check (#25475)
add a383a2bc83a [cases](regresstest)add json format regress test for
nested types (#25397)
add 1130317b91d [Improvement](statistics)Collect stats for hive partition
column using metadata (#24853)
add 5f844486e3b [enhancement](invert index) read columns by index reduce
seek time (#24735)
add 384fddb2ffa [test](case)add some debug log in mv case (#25458)
add 410441b5162 [enhancement](Nereids): remove LAsscom in Bushy Tree
RuleSet (#25465)
add c2fe34dec78 [refine](pipelineX) refactor local state (#25448)
add 0ee06f30b02 [feature](nereids)Ignore some node in 'explain shape plan'
command (#25485)
add c4cc6cefda9 [fix](regression-test) fix http stream 2pc case(#25507)
add 4d12d8885ef [feature](Nereids): graphSimplifier should compare
edge1BeforeEdge2 and edge2BeforeEdge1 (#25416)
add 1514f78b87e [refactor](partial-update) Split partial update infos from
tablet schema (#25147)
add 8c5af5a088a [fix](case) Fix test_analyze case (#25476)
add 652d6c57c03 [fix](jdbc catalog) fix handle oracle date format (#25487)
add 31a5e072e76 [refactor](pipelineX) Simplify set operation (#25502)
add f38f5f50eb1 [fix](ipv6)fix can not resolve host and port (#25254)
add 06ff59bc03d [Performance](sink) SIMD the tablet sink valied data
function (#25480)
add af8832389fe [feature](Nereids) add 4 array functions (#25488)
add 9d6b2dceb29 [fix](Nereids) non-slot filter should not be push through
aggregate (#25525)
add 8eff1486bdb [feature](nereids)print query id with memo and physical
tree (#25501)
add b74836050ad [chore](config) turnoff fuzzy for `enable_simdjson_reader`
(#25521)
add 9b1cdd3230f [fix](planner) mark join slot should always be nullable
(#25433)
add d287f53d770 [fix](nereids)in physical plan, print join class simple
name not full name #25515
add f6f1e3b6463 [chore](build) Bump the version of hyperscan (#25464)
add ce18f1148a5 [improvement](catalog)compatible with paimon 0.5 (#24985)
add 18c2a13e09c [fix](multi-catalog)fix maxcompute partition filter and
session creation (#24911)
add b76e23fb34e [improvement](meta) allow to ignore unknown image module
(#25450)
add 26e332c6084 [fix](multi-catalog)add exception for unsupported hive
input format (#25490)
add 47689fd4524 [refactor](jni) unified jni framework for java udf (#25302)
add 7cfb1d9b0e1 [Regression case](statistics) Add regression test case for
fetching HMSExternalTable through hms. (#25548)
add 32254952337 [regression-test](export) Add some tests that use hive
external table to read orc/parquet file exported by doris (#25431)
add 5503d04be29 [fix](test) create table should with distribution info
(#25544)
new 032555667fb merge block to single block on join/set node
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8b6c0951c6d)
\
N -- N -- N refs/heads/dev_join (032555667fb)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/CMakeLists.txt | 2 +-
be/src/agent/be_exec_version_manager.h | 4 +-
be/src/agent/cgroup_cpu_ctl.cpp | 8 +-
be/src/agent/utils.cpp | 12 +-
be/src/common/config.cpp | 7 +-
be/src/common/config.h | 2 +
be/src/common/status.h | 8 +-
be/src/http/action/pad_rowset_action.cpp | 3 +-
be/src/io/cache/block/block_file_cache.cpp | 8 +-
be/src/io/cache/block/block_file_cache.h | 20 +-
be/src/io/cache/block/block_file_segment.cpp | 30 +
be/src/io/cache/block/block_file_segment.h | 4 +
be/src/io/cache/block/block_lru_file_cache.cpp | 17 +
be/src/io/cache/block/block_lru_file_cache.h | 3 +
be/src/io/fs/benchmark/fs_benchmark_tool.cpp | 10 +-
be/src/io/fs/s3_file_bufferpool.cpp | 336 ++++++
be/src/io/fs/s3_file_bufferpool.h | 356 ++++++
be/src/io/fs/s3_file_write_bufferpool.cpp | 109 --
be/src/io/fs/s3_file_write_bufferpool.h | 150 ---
be/src/io/fs/s3_file_writer.cpp | 168 ++-
be/src/io/fs/s3_file_writer.h | 15 +-
be/src/io/io_common.h | 2 +
be/src/olap/cold_data_compaction.cpp | 2 +-
be/src/olap/compaction.cpp | 10 +-
be/src/olap/data_dir.cpp | 25 +-
be/src/olap/data_dir.h | 8 +-
be/src/olap/delta_writer.cpp | 1 +
be/src/olap/delta_writer_v2.cpp | 10 +-
be/src/olap/delta_writer_v2.h | 3 +
be/src/olap/memtable.cpp | 20 +-
be/src/olap/memtable.h | 6 +-
be/src/olap/memtable_writer.cpp | 10 +-
be/src/olap/memtable_writer.h | 4 +
be/src/olap/olap_server.cpp | 8 +-
be/src/olap/partial_update_info.h | 54 +
be/src/olap/push_handler.cpp | 14 +-
be/src/olap/rowset/beta_rowset_writer.cpp | 63 +-
be/src/olap/rowset/beta_rowset_writer.h | 10 +-
be/src/olap/rowset/beta_rowset_writer_v2.h | 12 +-
be/src/olap/rowset/rowset_writer.h | 10 +-
be/src/olap/rowset/rowset_writer_context.h | 5 +
be/src/olap/rowset/segment_v2/binary_dict_page.cpp | 1 +
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 41 +-
be/src/olap/rowset/segment_v2/segment_writer.cpp | 20 +-
be/src/olap/rowset_builder.cpp | 21 +-
be/src/olap/rowset_builder.h | 7 +
be/src/olap/schema_change.cpp | 6 +-
be/src/olap/snapshot_manager.cpp | 7 +-
be/src/olap/tablet.cpp | 61 +-
be/src/olap/tablet.h | 7 +-
be/src/olap/tablet_schema.cpp | 68 +-
be/src/olap/tablet_schema.h | 24 +-
be/src/olap/txn_manager.cpp | 21 +-
be/src/olap/txn_manager.h | 5 +-
be/src/pipeline/exec/data_queue.h | 1 +
be/src/pipeline/exec/hashjoin_probe_operator.cpp | 5 +-
.../pipeline/exec/multi_cast_data_stream_sink.cpp | 13 +
be/src/pipeline/exec/multi_cast_data_stream_sink.h | 9 +-
.../exec/multi_cast_data_stream_source.cpp | 2 +-
be/src/pipeline/exec/set_probe_sink_operator.cpp | 6 +-
be/src/pipeline/exec/set_probe_sink_operator.h | 5 +-
be/src/pipeline/exec/set_sink_operator.cpp | 6 +-
be/src/pipeline/exec/set_sink_operator.h | 6 +-
be/src/pipeline/exec/set_source_operator.cpp | 13 +
be/src/pipeline/exec/set_source_operator.h | 2 +-
be/src/pipeline/exec/union_sink_operator.cpp | 16 +-
be/src/pipeline/exec/union_source_operator.cpp | 25 +-
be/src/pipeline/exec/union_source_operator.h | 4 +-
be/src/pipeline/pipeline_x/dependency.h | 32 +-
be/src/pipeline/pipeline_x/operator.cpp | 121 +-
be/src/pipeline/pipeline_x/operator.h | 12 +-
be/src/pipeline/pipeline_x/pipeline_x_task.cpp | 26 +-
be/src/runtime/descriptors.cpp | 1 +
be/src/runtime/descriptors.h | 2 +
be/src/runtime/exec_env.h | 3 +
be/src/runtime/exec_env_init.cpp | 11 +-
be/src/runtime/group_commit_mgr.cpp | 2 +-
be/src/service/backend_service.cpp | 2 +-
be/src/service/doris_main.cpp | 1 -
.../aggregate_function_count_old.cpp | 51 -
.../aggregate_function_count_old.h | 280 -----
.../aggregate_function_simple_factory.cpp | 4 -
.../aggregate_function_sum_old.cpp | 33 -
.../aggregate_function_sum_old.h | 201 ----
be/src/vec/columns/column_fixed_length_object.h | 17 +
.../vec/common/hash_table/hash_table_set_build.h | 54 +-
.../vec/common/hash_table/hash_table_set_probe.h | 52 +-
be/src/vec/exec/format/avro/avro_jni_reader.cpp | 2 +-
be/src/vec/exec/format/jni_reader.cpp | 2 +-
.../exec/format/table/max_compute_jni_reader.cpp | 3 +-
be/src/vec/exec/jni_connector.cpp | 409 ++++---
be/src/vec/exec/jni_connector.h | 137 +--
be/src/vec/exec/join/vhash_join_node.cpp | 4 +-
be/src/vec/exec/scan/vfile_scanner.cpp | 5 +-
be/src/vec/exec/vjdbc_connector.cpp | 41 +-
be/src/vec/exec/vset_operation_node.cpp | 2 +-
be/src/vec/exec/vset_operation_node.h | 3 +
.../vec/functions/array/function_array_index.cpp | 4 +
be/src/vec/functions/array/function_array_index.h | 37 +-
.../function_date_or_datetime_computation.cpp | 6 -
.../function_date_or_datetime_computation.h | 2 -
.../function_date_or_datetime_computation_v2.cpp | 7 -
be/src/vec/functions/function_java_udf.cpp | 319 +----
be/src/vec/functions/function_java_udf.h | 13 -
be/src/vec/functions/function_map.cpp | 13 +-
be/src/vec/functions/function_string.cpp | 6 +-
be/src/vec/functions/function_string.h | 59 +-
be/src/vec/functions/simple_function_factory.h | 14 +-
be/src/vec/functions/time_of_function.cpp | 45 -
be/src/vec/functions/to_time_function.cpp | 62 -
be/src/vec/sink/vtablet_block_convertor.cpp | 173 +--
be/src/vec/sink/vtablet_block_convertor.h | 12 +-
be/src/vec/sink/vtablet_sink_v2.cpp | 5 +-
be/src/vec/sink/writer/vtablet_writer.cpp | 13 +-
be/test/io/fs/remote_file_system_test.cpp | 4 +-
be/test/io/fs/s3_file_writer_test.cpp | 479 ++++++++
be/test/olap/ordered_data_compaction_test.cpp | 3 +-
be/test/olap/rowid_conversion_test.cpp | 6 +-
be/test/olap/segcompaction_test.cpp | 10 +-
be/test/vec/olap/vertical_compaction_test.cpp | 21 +-
.../docker-compose/hive/scripts/README | 5 +
.../docker-compose/hive/scripts/hive-metastore.sh | 17 +
docs/en/docs/lakehouse/file.md | 6 +
docs/en/docs/query-acceleration/statistics.md | 5 +-
docs/en/docs/releasenotes/release-2.0.2.md | 157 +++
.../sql-functions/table-functions/hdfs.md | 2 +-
.../sql-manual/sql-functions/table-functions/s3.md | 1 +
docs/sidebars.json | 1 +
docs/zh-CN/docs/lakehouse/file.md | 6 +
docs/zh-CN/docs/query-acceleration/statistics.md | 5 +-
docs/zh-CN/docs/releasenotes/release-2.0.2.md | 210 ++++
.../sql-functions/table-functions/hdfs.md | 2 +-
.../sql-manual/sql-functions/table-functions/s3.md | 1 +
.../org/apache/doris/common/jni/JniScanner.java | 4 +-
.../org/apache/doris/common/jni/utils/OffHeap.java | 44 +-
.../doris/common/jni/utils/TypeNativeBytes.java | 192 ++-
.../apache/doris/common/jni/utils/UdfUtils.java | 4 +-
.../apache/doris/common/jni/vec/ColumnType.java | 21 +-
.../doris/common/jni/vec/ColumnValueConverter.java | 17 +-
.../apache/doris/common/jni/vec/VectorColumn.java | 820 ++++++++++++-
.../apache/doris/common/jni/vec/VectorTable.java | 140 ++-
.../apache/doris/common/jni/JniScannerTest.java | 8 +-
.../java/org/apache/doris/udf/UdfExecutor.java | 249 ++--
.../java/org/apache/doris/jdbc/JdbcExecutor.java | 11 +-
.../doris/maxcompute/MaxComputeJniScanner.java | 39 +-
.../doris/maxcompute/MaxComputePartitionValue.java | 137 +++
.../doris/maxcompute/MaxComputeTableScan.java | 30 +-
fe/be-java-extensions/paimon-scanner/pom.xml | 47 +-
.../org/apache/doris/paimon/PaimonJniScanner.java | 2 +-
.../org/apache/doris/paimon/PaimonTypeUtils.java | 229 ++++
fe/check/checkstyle/checkstyle.xml | 3 +
.../main/java/org/apache/doris/common/Config.java | 18 +
fe/fe-core/pom.xml | 9 +-
fe/fe-core/src/main/cup/sql_parser.cup | 10 +-
.../java/org/apache/doris/alter/RollupJobV2.java | 23 +-
.../org/apache/doris/alter/SchemaChangeJobV2.java | 4 +-
.../java/org/apache/doris/alter/SystemHandler.java | 6 +-
.../analysis/AdminCancelRebalanceDiskStmt.java | 4 +-
.../apache/doris/analysis/AdminCleanTrashStmt.java | 5 +-
.../doris/analysis/AdminRebalanceDiskStmt.java | 5 +-
.../apache/doris/analysis/AnalyzeProperties.java | 7 +-
.../org/apache/doris/analysis/AnalyzeStmt.java | 4 +
.../java/org/apache/doris/analysis/Analyzer.java | 5 +-
.../doris/analysis/CreateMaterializedViewStmt.java | 45 +-
.../main/java/org/apache/doris/analysis/Expr.java | 6 +
.../apache/doris/analysis/ExpressionFunctions.java | 2 +-
.../apache/doris/analysis/ShowTrashDiskStmt.java | 3 +-
.../org/apache/doris/analysis/StmtRewriter.java | 2 +-
.../java/org/apache/doris/catalog/BrokerMgr.java | 7 +-
.../doris/catalog/BuiltinScalarFunctions.java | 8 +
.../main/java/org/apache/doris/catalog/Env.java | 6 +-
.../java/org/apache/doris/catalog/FsBroker.java | 3 +-
.../java/org/apache/doris/catalog/OlapTable.java | 2 +-
.../java/org/apache/doris/catalog/Replica.java | 36 +-
.../apache/doris/catalog/TabletInvertedIndex.java | 22 +-
.../doris/catalog/external/HMSExternalTable.java | 10 +-
.../catalog/external/MaxComputeExternalTable.java | 80 ++
.../clone/ColocateTableCheckerAndBalancer.java | 523 ++++++++-
.../org/apache/doris/clone/TabletSchedCtx.java | 7 +
.../org/apache/doris/clone/TabletScheduler.java | 2 +-
.../org/apache/doris/common/proc/TrashProcDir.java | 3 +-
.../org/apache/doris/common/util/BrokerReader.java | 3 +-
.../org/apache/doris/common/util/BrokerUtil.java | 6 +-
.../datasource/MaxComputeExternalCatalog.java | 12 +-
.../doris/datasource/hive/HiveMetaStoreCache.java | 14 +-
.../doris/deploy/impl/AmbariDeployManager.java | 4 +-
.../apache/doris/external/hive/util/HiveUtil.java | 16 +
.../doris/fs/operations/BrokerFileOperations.java | 4 +-
.../doris/httpv2/controller/BaseController.java | 7 +-
.../doris/httpv2/rest/RestBaseController.java | 4 +-
.../doris/httpv2/rest/TableQueryPlanAction.java | 4 +-
.../doris/httpv2/rest/manager/ClusterAction.java | 10 +-
.../doris/httpv2/rest/manager/NodeAction.java | 35 +-
.../httpv2/rest/manager/QueryProfileAction.java | 4 +-
.../apache/doris/journal/bdbje/BDBJEJournal.java | 8 +-
.../java/org/apache/doris/ldap/LdapClient.java | 7 +-
.../java/org/apache/doris/master/MasterImpl.java | 9 +-
.../org/apache/doris/master/ReportHandler.java | 46 +-
.../java/org/apache/doris/metric/MetricRepo.java | 7 +-
.../java/org/apache/doris/mysql/MysqlChannel.java | 4 +-
.../org/apache/doris/nereids/NereidsPlanner.java | 5 +-
.../nereids/analyzer/PlaceholderExpression.java | 85 ++
.../exceptions/DialectTransformException.java | 19 +-
.../exceptions/UnsupportedDialectException.java} | 21 +-
.../glue/translator/PhysicalPlanTranslator.java | 3 +-
.../glue/translator/RuntimeFilterTranslator.java | 4 +-
.../nereids/jobs/joinorder/hypergraph/Edge.java | 8 +-
.../jobs/joinorder/hypergraph/GraphSimplifier.java | 20 +-
.../jobs/joinorder/hypergraph/HyperGraph.java | 17 +-
.../doris/nereids/parser/LogicalPlanBuilder.java | 63 +-
.../parser/LogicalPlanBuilderAssistant.java | 111 ++
.../apache/doris/nereids/parser/NereidsParser.java | 61 +
.../apache/doris/nereids/parser/ParseDialect.java | 111 ++
.../apache/doris/nereids/parser/ParserContext.java | 22 +-
.../parser/trino/AbstractFnCallTransformer.java | 46 +
.../trino/ComplexTrinoFnCallTransformer.java | 16 +-
.../parser/trino/DateDiffFnCallTransformer.java | 66 ++
.../parser/trino/LogicalPlanTrinoBuilder.java | 325 ++++++
.../parser/trino/TrinoFnCallTransformer.java | 118 ++
.../parser/trino/TrinoFnCallTransformers.java | 130 +++
.../doris/nereids/parser/trino/TrinoParser.java} | 21 +-
.../processor/post/RuntimeFilterGenerator.java | 132 ++-
.../org/apache/doris/nereids/rules/RuleSet.java | 2 -
.../org/apache/doris/nereids/rules/RuleType.java | 12 +-
.../join/InnerJoinLeftAssociateProject.java | 2 +-
.../join/InnerJoinRightAssociateProject.java | 2 +-
.../exploration/join/JoinExchangeBothProject.java | 2 +-
.../exploration/join/JoinExchangeLeftProject.java | 2 +-
.../exploration/join/JoinExchangeRightProject.java | 2 +-
.../join/LogicalJoinSemiJoinTranspose.java | 4 +-
.../join/LogicalJoinSemiJoinTransposeProject.java | 4 +-
.../join/PushdownProjectThroughInnerOuterJoin.java | 8 +-
.../join/PushdownProjectThroughSemiJoin.java | 8 +-
.../rewrite/PushdownFilterThroughAggregation.java | 8 +-
.../mv/AbstractSelectMaterializedIndexRule.java | 8 -
.../mv/SelectMaterializedIndexWithAggregate.java | 36 +-
.../scalar/{CountEqual.java => ArrayConcat.java} | 28 +-
.../functions/scalar/ArrayContains.java | 8 +-
.../functions/scalar/ArrayPosition.java | 8 +-
.../scalar/{CountEqual.java => ArrayPushBack.java} | 25 +-
.../{CountEqual.java => ArrayPushFront.java} | 25 +-
.../expressions/functions/scalar/ArrayZip.java | 91 ++
.../expressions/functions/scalar/CountEqual.java | 8 +-
.../functions/scalar/MapContainsKey.java | 8 +-
.../functions/scalar/MapContainsValue.java | 8 +-
.../expressions/visitor/ExpressionVisitor.java | 9 +
.../expressions/visitor/ScalarFunctionVisitor.java | 20 +
.../org/apache/doris/nereids/trees/plans/Plan.java | 10 +-
.../trees/plans/physical/AbstractPhysicalJoin.java | 28 +
.../trees/plans/physical/PhysicalHashJoin.java | 29 -
.../plans/physical/PhysicalNestedLoopJoin.java | 23 +-
.../trees/plans/physical/RuntimeFilter.java | 22 +-
.../org/apache/doris/persist/meta/MetaReader.java | 16 +-
.../org/apache/doris/planner/DataStreamSink.java | 19 +-
.../org/apache/doris/planner/JoinNodeBase.java | 1 +
.../java/org/apache/doris/planner/PlanNode.java | 19 +-
.../org/apache/doris/planner/RuntimeFilter.java | 66 +-
.../doris/planner/external/MaxComputeScanNode.java | 4 +-
.../doris/planner/external/jdbc/JdbcScanNode.java | 3 +-
.../java/org/apache/doris/qe/SessionVariable.java | 47 +
.../java/org/apache/doris/qe/ShowExecutor.java | 6 +-
.../org/apache/doris/statistics/AnalysisInfo.java | 8 +-
.../doris/statistics/AnalysisInfoBuilder.java | 20 +-
.../apache/doris/statistics/AnalysisManager.java | 1 +
.../apache/doris/statistics/BaseAnalysisTask.java | 18 +
.../apache/doris/statistics/HMSAnalysisTask.java | 154 ++-
.../doris/statistics/util/StatisticsUtil.java | 5 +
.../org/apache/doris/system/SystemInfoService.java | 13 +-
.../ExternalFileTableValuedFunction.java | 56 +-
.../java/org/apache/doris/task/AgentBatchTask.java | 7 +-
fe/fe-core/src/main/jflex/sql_scanner.flex | 2 +
.../org/apache/doris/catalog/CatalogTestUtil.java | 44 +
.../org/apache/doris/clone/BalanceStatistic.java | 102 ++
.../ColocateTableCheckerAndBalancerPerfTest.java | 196 ++++
.../clone/ColocateTableCheckerAndBalancerTest.java | 112 +-
.../org/apache/doris/clone/RebalancerTestUtil.java | 48 +-
.../org/apache/doris/clone/RepairVersionTest.java | 176 +++
.../joinorder/hypergraph/GraphSimplifierTest.java | 28 +-
.../org/apache/doris/nereids/memo/RankTest.java | 11 +-
.../doris/nereids/parser/NereidsParserTest.java | 19 +
.../doris/nereids/parser/ParserTestBase.java | 5 +
.../nereids/parser/trino/FnTransformTest.java | 46 +
.../{ParserTestBase.java => trino/QueryTest.java} | 29 +-
.../doris/nereids/util/HyperGraphBuilder.java | 12 +-
.../nereids/util/TrinoDialectPlanParseChecker.java | 59 +
.../apache/doris/utframe/MockedBackendFactory.java | 74 +-
.../apache/doris/utframe/TestWithFeService.java | 7 +-
.../org/apache/doris/utframe/UtFrameUtils.java | 5 +-
fe/pom.xml | 12 +-
gensrc/proto/descriptors.proto | 6 +-
gensrc/proto/olap_file.proto | 4 +-
gensrc/script/doris_builtins_functions.py | 114 +-
gensrc/thrift/Descriptors.thrift | 1 +
regression-test/conf/regression-conf.groovy | 7 +-
.../data/correctness_p0/test_mv_case.out | 7 +
.../test_outer_join_with_inline_view.out | 3 +
.../test_nestedtypes_json_insert_into_with_s3.out | 139 +++
.../export/hive_read/orc/test_hive_read_orc.out | 49 +
.../orc/test_hive_read_orc_complex_type.out | 124 ++
.../hive_read/parquet/test_hive_read_parquet.out | 49 +
.../parquet/test_hive_read_parquet_comlex_type.out | 124 ++
.../external_table_p0/hive/test_hive_other.out | 5 -
.../hive/test_prepare_hive_data_in_case.out | 11 +
.../jdbc/test_oracle_jdbc_catalog.out | 3 +
.../paimon/test_paimon_catalog.out | 68 ++
.../tvf/test_hdfs_tvf_error_uri.out | 6 +
.../test_external_catalog_maxcompute.out | 24 +
.../data/javaudf_p0/test_javaudf_all_types.out | 13 +
.../load_p0/stream_load/invalid_nest_json2.json | 2 +-
.../data/load_p0/tvf/test_tvf_empty_file.out | 17 +
.../data/load_p0/tvf/test_tvf_error_url.out | 11 +
.../multiple_ssb_between.out} | 0
.../nereids_function_p0/scalar_function/Array.out | 1224 +++++++++++++++++++-
.../eliminate_outer_join/eliminate_outer_join.out | 195 ++--
.../filter_push_through_aggregate.out | 3 +
.../data/nereids_ssb_shape_sf100_p0/shape/q2.1.out | 21 +-
.../data/nereids_ssb_shape_sf100_p0/shape/q4.1.out | 39 +-
.../nereids_tpcds_shape_sf100_p0/shape/query11.out | 23 +-
.../nereids_tpcds_shape_sf100_p0/shape/query21.out | 19 +-
.../nereids_tpcds_shape_sf100_p0/shape/query23.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query26.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query27.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query32.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query34.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query36.out | 25 +-
.../nereids_tpcds_shape_sf100_p0/shape/query37.out | 15 +-
.../nereids_tpcds_shape_sf100_p0/shape/query39.out | 17 +-
.../nereids_tpcds_shape_sf100_p0/shape/query43.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query44.out | 57 +-
.../nereids_tpcds_shape_sf100_p0/shape/query46.out | 18 +-
.../nereids_tpcds_shape_sf100_p0/shape/query50.out | 14 +-
.../nereids_tpcds_shape_sf100_p0/shape/query65.out | 55 +-
.../nereids_tpcds_shape_sf100_p0/shape/query67.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query68.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query7.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query70.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query73.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query75.out | 39 +-
.../nereids_tpcds_shape_sf100_p0/shape/query79.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query8.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query81.out | 10 +-
.../nereids_tpcds_shape_sf100_p0/shape/query82.out | 15 +-
.../nereids_tpcds_shape_sf100_p0/shape/query89.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query92.out | 13 +-
.../nereids_tpcds_shape_sf100_p0/shape/query99.out | 8 +-
.../nereids_tpch_shape_sf1000_p0/shape/q21.out | 10 +-
.../array_functions/test_array_with_scale_type.out | 32 +-
.../org/apache/doris/regression/suite/Suite.groovy | 48 +-
.../src/main/java/org/apache/doris/udf/Echo.java | 137 +++
.../java/org/apache/doris/udf/MapStrStrTest.java | 6 +-
.../suites/correctness_p0/test_mv_case.groovy | 41 +
.../test_outer_join_with_inline_view.groovy | 23 +
.../correctness_p0/test_runtime_filter.groovy | 111 ++
...est_nestedtypes_json_insert_into_with_s3.groovy | 253 ++++
.../export/hive_read/orc/test_hive_read_orc.groovy | 272 +++++
.../orc/test_hive_read_orc_complex_type.groovy | 280 +++++
.../parquet/test_hive_read_parquet.groovy | 272 +++++
.../test_hive_read_parquet_comlex_type.groovy | 282 +++++
.../external_table_p0/hive/test_hive_other.groovy | 13 +-
.../hive/test_prepare_hive_data_in_case.groovy | 11 +-
.../jdbc/test_oracle_jdbc_catalog.groovy | 1 +
.../paimon/test_paimon_catalog.groovy | 61 +
.../tvf/test_hdfs_tvf_error_uri.groovy | 43 +
.../test_insert_from_tvf_with_common_user.groovy | 2 +-
.../hive/test_external_catalog_hive.groovy | 7 +
.../hive/test_hive_partition_column_analyze.groovy | 230 ++++
.../hive/test_hive_statistics_from_hms.groovy | 214 ++++
.../test_external_catalog_maxcompute.groovy | 60 +
.../javaudf_p0/test_javaudf_all_types.groovy | 236 ++++
.../test_group_commit_http_stream.groovy | 8 +-
.../http_stream/test_http_stream_2pc.groovy | 15 +
.../test_group_commit_stream_load.groovy | 8 +-
.../suites/load_p0/tvf/test_tvf_empty_file.groovy | 69 ++
.../suites/load_p0/tvf/test_tvf_error_url.groovy | 61 +
.../suites/mv_p0/null_insert/null_insert.groovy | 2 +-
.../multiple_ssb_between.groovy | 244 ++++
.../scalar_function/Array.groovy | 108 +-
.../eliminate_outer_join.groovy | 4 +-
.../filter_push_through_aggregate.groovy | 29 +-
.../nereids_tpcds_shape_sf100_p0/rf/ds_rf46.groovy | 3 +-
.../nereids_tpcds_shape_sf100_p0/rf/ds_rf50.groovy | 3 +-
.../nereids_tpcds_shape_sf100_p0/rf/ds_rf99.groovy | 3 +-
.../nereids_tpch_shape_sf1000_p0/rf/h_rf21.groovy | 2 +-
.../test_array_with_scale_type.groovy | 16 +-
.../suites/statistics/analyze_stats.groovy | 38 +-
thirdparty/download-thirdparty.sh | 9 +-
thirdparty/patches/hyperscan-5.4.0.patch | 18 -
thirdparty/vars.sh | 8 +-
388 files changed, 13915 insertions(+), 3724 deletions(-)
create mode 100644 be/src/io/fs/s3_file_bufferpool.cpp
create mode 100644 be/src/io/fs/s3_file_bufferpool.h
delete mode 100644 be/src/io/fs/s3_file_write_bufferpool.cpp
delete mode 100644 be/src/io/fs/s3_file_write_bufferpool.h
create mode 100644 be/src/olap/partial_update_info.h
delete mode 100644
be/src/vec/aggregate_functions/aggregate_function_count_old.cpp
delete mode 100644
be/src/vec/aggregate_functions/aggregate_function_count_old.h
delete mode 100644
be/src/vec/aggregate_functions/aggregate_function_sum_old.cpp
delete mode 100644 be/src/vec/aggregate_functions/aggregate_function_sum_old.h
create mode 100644 be/test/io/fs/s3_file_writer_test.cpp
create mode 100644 docs/en/docs/releasenotes/release-2.0.2.md
create mode 100644 docs/zh-CN/docs/releasenotes/release-2.0.2.md
copy be/src/vec/functions/array/function_array_index.cpp =>
fe/be-java-extensions/java-common/src/main/java/org/apache/doris/common/jni/vec/ColumnValueConverter.java
(63%)
create mode 100644
fe/be-java-extensions/max-compute-scanner/src/main/java/org/apache/doris/maxcompute/MaxComputePartitionValue.java
create mode 100644
fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonTypeUtils.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/PlaceholderExpression.java
copy be/src/vec/functions/array/function_array_index.cpp =>
fe/fe-core/src/main/java/org/apache/doris/nereids/exceptions/DialectTransformException.java
(63%)
copy
fe/fe-core/src/{test/java/org/apache/doris/nereids/parser/ParserTestBase.java
=>
main/java/org/apache/doris/nereids/exceptions/UnsupportedDialectException.java}
(54%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilderAssistant.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/ParseDialect.java
copy be/src/vec/functions/array/function_array_index.cpp =>
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/ParserContext.java
(63%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/trino/AbstractFnCallTransformer.java
copy be/src/vec/functions/array/function_array_index.cpp =>
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/trino/ComplexTrinoFnCallTransformer.java
(63%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/trino/DateDiffFnCallTransformer.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/trino/LogicalPlanTrinoBuilder.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/trino/TrinoFnCallTransformer.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/trino/TrinoFnCallTransformers.java
copy
fe/fe-core/src/{test/java/org/apache/doris/nereids/parser/ParserTestBase.java
=> main/java/org/apache/doris/nereids/parser/trino/TrinoParser.java} (58%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{CountEqual.java
=> ArrayConcat.java} (66%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{CountEqual.java
=> ArrayPushBack.java} (68%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{CountEqual.java
=> ArrayPushFront.java} (68%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/ArrayZip.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/clone/BalanceStatistic.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/clone/ColocateTableCheckerAndBalancerPerfTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/clone/RepairVersionTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/parser/trino/FnTransformTest.java
copy
fe/fe-core/src/test/java/org/apache/doris/nereids/parser/{ParserTestBase.java
=> trino/QueryTest.java} (50%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/util/TrinoDialectPlanParseChecker.java
create mode 100644 regression-test/data/correctness_p0/test_mv_case.out
create mode 100644
regression-test/data/datatype_p0/nested_types/query/test_nestedtypes_json_insert_into_with_s3.out
create mode 100644
regression-test/data/external_table_p0/export/hive_read/orc/test_hive_read_orc.out
create mode 100644
regression-test/data/external_table_p0/export/hive_read/orc/test_hive_read_orc_complex_type.out
create mode 100644
regression-test/data/external_table_p0/export/hive_read/parquet/test_hive_read_parquet.out
create mode 100644
regression-test/data/external_table_p0/export/hive_read/parquet/test_hive_read_parquet_comlex_type.out
create mode 100644
regression-test/data/external_table_p0/hive/test_prepare_hive_data_in_case.out
create mode 100644
regression-test/data/external_table_p0/paimon/test_paimon_catalog.out
create mode 100644
regression-test/data/external_table_p0/tvf/test_hdfs_tvf_error_uri.out
create mode 100644
regression-test/data/external_table_p2/maxcompute/test_external_catalog_maxcompute.out
create mode 100644 regression-test/data/javaudf_p0/test_javaudf_all_types.out
create mode 100644 regression-test/data/load_p0/tvf/test_tvf_empty_file.out
create mode 100644 regression-test/data/load_p0/tvf/test_tvf_error_url.out
copy regression-test/data/mv_p0/ssb/{multiple_ssb/multiple_ssb.out =>
multiple_ssb_between/multiple_ssb_between.out} (100%)
create mode 100644
regression-test/data/nereids_rules_p0/filter_push_down/filter_push_through_aggregate.out
create mode 100644
regression-test/java-udf-src/src/main/java/org/apache/doris/udf/Echo.java
create mode 100644 regression-test/suites/correctness_p0/test_mv_case.groovy
create mode 100644
regression-test/suites/correctness_p0/test_runtime_filter.groovy
create mode 100644
regression-test/suites/datatype_p0/nested_types/query/test_nestedtypes_json_insert_into_with_s3.groovy
create mode 100644
regression-test/suites/external_table_p0/export/hive_read/orc/test_hive_read_orc.groovy
create mode 100644
regression-test/suites/external_table_p0/export/hive_read/orc/test_hive_read_orc_complex_type.groovy
create mode 100644
regression-test/suites/external_table_p0/export/hive_read/parquet/test_hive_read_parquet.groovy
create mode 100644
regression-test/suites/external_table_p0/export/hive_read/parquet/test_hive_read_parquet_comlex_type.groovy
create mode 100644
regression-test/suites/external_table_p0/tvf/test_hdfs_tvf_error_uri.groovy
create mode 100644
regression-test/suites/external_table_p2/hive/test_hive_partition_column_analyze.groovy
create mode 100644
regression-test/suites/external_table_p2/hive/test_hive_statistics_from_hms.groovy
create mode 100644
regression-test/suites/external_table_p2/maxcompute/test_external_catalog_maxcompute.groovy
create mode 100644
regression-test/suites/javaudf_p0/test_javaudf_all_types.groovy
create mode 100644
regression-test/suites/load_p0/tvf/test_tvf_empty_file.groovy
create mode 100644 regression-test/suites/load_p0/tvf/test_tvf_error_url.groovy
create mode 100644
regression-test/suites/mv_p0/ssb/multiple_ssb_between/multiple_ssb_between.groovy
copy be/src/vec/functions/array/function_array_index.cpp =>
regression-test/suites/nereids_rules_p0/filter_push_down/filter_push_through_aggregate.groovy
(51%)
delete mode 100644 thirdparty/patches/hyperscan-5.4.0.patch
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
