This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch auto-pick-66706-branch-4.1
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 8333725b48a [fix](compaction) Use row count for cumulative task
classification (#66706)
add bfafcc7410d Update github host from macos-13 to macos-15 (#67746)
add 8fa99196dc8 branch-4.1: [perf](load) Optimize write performance with
improved backpressure control (#67728)
add b85792bb113 branch-4.1: [fix](be) Initialize bthread context key
before first use (#67722)
add 04aea05a103 branch-4.1: [fix](cloud) Return existing rowset for
concurrent commits #67637 (#67660)
add db2aceff2c8 branch-4.1: [fix](regression) Deflake the sql cache
regression suites #67734 (#67735)
add 7ed09f49eaf [fix](iceberg) Support fractional timestamps in time
travel (#67704)
add a8344169095 branch-4.1: [fix](iceberg) Run execute actions in catalog
auth scope (#67695)
add a7e9c5ffeda branch-4.1: [fix](rpc) Fix AutoReleaseClosure data race
with callback reuse (#61782) (#67340)
add 55f38f22789 branch-4.1: [fix](iceberg) Fix V2 reads across nested
schema evolution #67574 (#67690)
add 854496685d3 branch-4.1: [feature](lance) Durable Lance index job
infrastructure with fence, quota and replay (#67235)
add 14d3d957c02 branch-4.1: [fix](group commit) Fix group commit routing
for virtual compute groups #66585 (#67590)
add 95265876df7 branch-4.1: [improvement](fe) Improve audit logs for S3
streaming insert jobs #67489 (#67756)
add 03b6cc30fb0 branch-4.1: [fix](cloud) Exclude covered rowsets from
compaction minimum timestamps #67617 (#67751)
add 09ef7468564 branch-4.1: [fix](p2) fix p2 regression failures #67731
(#67760)
add 33add61d069 branch-4.1: [Enhance](resource) Restrict modification of
root-created AI resources (#67782)
add fd32cd8c682 branch-4.1: [Fix](ai_func) Parse final text from Responses
API output (#67671) (#67795)
add a060d016459 branch-4.1: [Fix](test) make pyudf ut deterministic and
meaningful #67352 (#67806)
add 2776fc95d35 branch-4.1:[fix](build) Fix ASAN getentropy crash on glibc
2.17 (#67849)
add 5fcda491a19 branch-4.1: [fix](fe) Preserve active export jobs and
label mappings during cleanup (#67539) (#67833)
add f04008b56d2 branch-4.1: [fix](expr opt) Preserve cast-induced
nullability in comparisons #67730 (#67843)
add 00fdd510f51 branch-4.1: [fix](SVGuard) Preserve session guards during
common expression extraction #67717 (#67863)
add a8c7c355250 branch-4.1: [fix](agg) Preserve weighted average when
eliminating group by #67638 (#67832)
add 88864d93e62 branch-4.1: [fix](tvf) Preserve duplicate output column
positions #67769 (#67829)
add 5cf524587c6 branch-4.1: [fix](regression) Stop MTMV task waits from
latching onto the previous task #67710 (#67821)
add 59b60ba85f0 branch-4.1: [fix](fe) Upgrade Nimbus dependencies #67065
(#67847)
add a68e9ef933d branch-4.1: [improvement](cloud) Point delete versioned
delete bitmap keys when recycling rowsets (#67827) (#67854)
add 1b5d6b84771 branch-4.1: [improvement](cloud) Add blob key codec
helpers #67013 (#67838)
add 523ffce8171 [fix](compaction) Use row count for cumulative task
classification (#66706)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8333725b48a)
\
N -- N -- N refs/heads/auto-pick-66706-branch-4.1 (523ffce8171)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/build-thirdparty.yml | 2 +-
be/src/cloud/cloud_delta_writer.cpp | 16 +-
be/src/cloud/cloud_rowset_builder.cpp | 8 +
be/src/cloud/cloud_rowset_builder.h | 2 +
be/src/exec/exchange/vdata_stream_sender.h | 1 +
be/src/exec/operator/exchange_sink_buffer.cpp | 3 +-
be/src/exec/runtime_filter/runtime_filter.cpp | 23 +-
be/src/exec/runtime_filter/runtime_filter.h | 7 +
be/src/exec/runtime_filter/runtime_filter_mgr.cpp | 73 +-
be/src/exec/runtime_filter/runtime_filter_mgr.h | 6 +
.../runtime_filter/runtime_filter_producer.cpp | 63 +-
.../exec/runtime_filter/runtime_filter_producer.h | 66 +-
be/src/exec/sink/writer/vtablet_writer.cpp | 1 +
be/src/exprs/aggregate/aggregate_function_ai_agg.h | 12 +-
be/src/exprs/function/ai/ai_adapter.h | 141 +-
be/src/format_v2/column_mapper.cpp | 107 +
be/src/format_v2/column_mapper.h | 4 +
be/src/format_v2/table/iceberg_reader.cpp | 71 +
be/src/format_v2/table/iceberg_reader.h | 8 +
be/src/format_v2/table/iceberg_schema_utils.h | 12 +
be/src/format_v2/table_reader.cpp | 2 +
be/src/format_v2/table_reader.h | 4 +-
be/src/glibc-compatibility/CMakeLists.txt | 13 +-
be/src/runtime/thread_context.cpp | 18 +
be/src/runtime/thread_context.h | 5 +
be/src/service/internal_service.cpp | 8 -
be/src/udf/python/python_server.cpp | 55 +-
be/src/udf/python/python_server.h | 8 +-
be/src/util/brpc_closure.h | 114 +-
be/test/ai/aggregate_function_ai_agg_test.cpp | 45 +-
be/test/ai/ai_adapter_test.cpp | 263 +-
.../runtime_filter/sync_size_callback_test.cpp | 351 +
be/test/format_v2/column_mapper_test.cpp | 38 +
be/test/format_v2/table/iceberg_reader_test.cpp | 251 +-
be/test/udf/python/python_env_test.cpp | 179 +-
be/test/udf/python/python_server_test.cpp | 398 +-
be/test/udf/python/python_udf_meta_test.cpp | 144 +-
be/test/udf/python/python_udf_runtime_test.cpp | 225 +-
cloud/src/meta-service/meta_service.cpp | 40 +-
cloud/src/meta-store/blob_message.cpp | 63 +-
cloud/src/meta-store/blob_message.h | 55 +-
cloud/src/meta-store/meta_reader.cpp | 13 +-
cloud/src/recycler/recycler.cpp | 88 +-
cloud/src/recycler/recycler.h | 10 +-
cloud/test/blob_message_test.cpp | 124 +
cloud/test/clone_chain_reader_test.cpp | 52 +
cloud/test/meta_reader_test.cpp | 130 +
cloud/test/meta_service_test.cpp | 45 +
cloud/test/recycler_operation_log_test.cpp | 179 +
cloud/test/recycler_test.cpp | 210 +-
cloud/test/schema_kv_test.cpp | 20 +-
cloud/test/txn_kv_test.cpp | 4 +-
.../iceberg_load/run01.sql | 10 +-
.../data/data.parquet | Bin 0 -> 1081 bytes
...aafe0-9de1-4b3f-942d-305dc33fb82f.metadata.json | 1 +
...09fba-61c6-4e4b-b0f5-6023f70d5033.metadata.json | 1 +
...3fffe-54f9-486a-b15e-7333fdb96f61.metadata.json | 1 +
...3e871-e421-48ed-a735-354f4e1c9502.metadata.json | 1 +
...544-1-94f693c5-6efb-4e9d-a40e-f02ac301fc14.avro | Bin 0 -> 4667 bytes
...ifest-d04009d1-83e2-4d2a-859e-79d57e8dd381.avro | Bin 0 -> 7501 bytes
.../java/org/apache/doris/catalog/AIResource.java | 10 +
.../main/java/org/apache/doris/catalog/Env.java | 25 +
.../java/org/apache/doris/catalog/Resource.java | 8 +
.../java/org/apache/doris/catalog/ResourceMgr.java | 7 +-
.../doris/cloud/system/CloudSystemInfoService.java | 41 +
.../iceberg/IcebergExternalMetaCache.java | 11 +-
.../doris/datasource/iceberg/IcebergUtils.java | 29 +-
.../iceberg/action/BaseIcebergAction.java | 14 +-
.../action/IcebergCherrypickSnapshotAction.java | 9 +-
.../action/IcebergExecuteActionFactory.java | 42 +-
.../action/IcebergExpireSnapshotsAction.java | 8 +-
.../iceberg/action/IcebergFastForwardAction.java | 8 +-
.../action/IcebergPublishChangesAction.java | 9 +-
.../action/IcebergRewriteDataFilesAction.java | 5 +-
.../action/IcebergRewriteManifestsAction.java | 8 +-
.../action/IcebergRollbackToSnapshotAction.java | 8 +-
.../action/IcebergRollbackToTimestampAction.java | 8 +-
.../action/IcebergSetCurrentSnapshotAction.java | 8 +-
.../lance/job/LanceIndexDatasetLocator.java | 143 +
.../datasource/lance/job/LanceIndexFenceKey.java | 92 +
.../doris/datasource/lance/job/LanceIndexJob.java | 699 ++
.../lance/job/LanceIndexJobCompletionReason.java | 28 +
.../datasource/lance/job/LanceIndexJobManager.java | 753 ++
.../lance/job/LanceIndexJobMutationState.java | 47 +
.../lance/job/LanceIndexJobMutationType.java | 28 +
.../datasource/lance/job/LanceIndexJobQuota.java | 189 +
.../lance/job/LanceIndexJobRefreshState.java | 41 +
.../datasource/lance/job/LanceIndexJobResult.java | 112 +
.../lance/job/LanceIndexJobResultCode.java | 158 +
.../lance/job/LanceIndexNameNormalizer.java | 69 +
.../lance/job/LanceIndexSchemaContract.java | 250 +
.../lance/job/LanceIndexTerminationProof.java | 33 +
.../insert/streaming/StreamingInsertTask.java | 51 +-
.../job/offset/s3/S3SourceOffsetProvider.java | 2 +-
.../org/apache/doris/journal/JournalEntity.java | 6 +
.../main/java/org/apache/doris/load/ExportMgr.java | 25 +-
.../org/apache/doris/load/GroupCommitManager.java | 60 +-
.../processor/post/CommonSubExpressionOpt.java | 19 +-
.../doris/nereids/rules/analysis/BindSink.java | 9 +-
.../rules/SimplifyComparisonPredicate.java | 25 +-
.../nereids/rules/rewrite/EliminateGroupBy.java | 4 +-
.../trees/plans/commands/AlterResourceCommand.java | 6 +-
.../plans/commands/CreateResourceCommand.java | 2 +-
.../trees/plans/commands/DropResourceCommand.java | 8 +
.../trees/plans/commands/ExecuteActionCommand.java | 75 +-
.../java/org/apache/doris/persist/EditLog.java | 10 +
.../org/apache/doris/persist/OperationType.java | 3 +
.../doris/persist/meta/MetaPersistMethod.java | 6 +
.../doris/persist/meta/PersistMetaModules.java | 2 +-
.../java/org/apache/doris/qe/SessionVariable.java | 3 +
.../org/apache/doris/catalog/AIResourceTest.java | 22 +-
.../cloud/system/CloudSystemInfoServiceTest.java | 8 +
.../doris/datasource/iceberg/IcebergUtilsTest.java | 33 +-
.../lance/job/LanceIndexDatasetLocatorTest.java | 183 +
.../lance/job/LanceIndexJobManagerPersistTest.java | 457 ++
.../lance/job/LanceIndexJobManagerReplayTest.java | 628 ++
.../lance/job/LanceIndexJobQuotaTest.java | 208 +
.../lance/job/LanceIndexJobResultClassifyTest.java | 239 +
.../lance/job/LanceIndexJobStateMachineTest.java | 640 ++
.../datasource/lance/job/LanceIndexJobTest.java | 303 +
.../lance/job/LanceIndexJobWiringTest.java | 78 +
.../lance/job/LanceIndexNameNormalizerTest.java | 111 +
.../streaming/StreamingInsertTaskAuditTest.java | 270 +
.../GroupCommitManagerBackendSelectionTest.java | 3 +-
.../apache/doris/load/GroupCommitManagerTest.java | 188 +
.../apache/doris/load/loadv2/ExportMgrTest.java | 142 +
.../postprocess/CommonSubExpressionTest.java | 105 +
.../rules/analysis/BindTVFTableSinkTest.java | 58 +
.../rules/SimplifyComparisonPredicateTest.java | 30 +
.../rewrite/EliminateGroupByAvgWeightedTest.java | 57 +
.../plans/commands/AIResourceCommandTest.java | 118 +
.../plans/commands/ExecuteActionCommandTest.java | 245 +
.../org/apache/doris/qe/SessionVariablesTest.java | 1 +
fe/pom.xml | 2 +-
.../three_level_nestedtypes_with_s3data.out | 7762 ++++++++++----------
.../two_level_nestedtypes_with_s3data.out | 2362 +++---
.../iceberg/test_gen_iceberg_by_api.out | 3 -
...ceberg_migrated_nested_without_name_mapping.out | 3 +
.../test_iceberg_struct_schema_evolution.out | 3 +
.../tvf/insert/test_insert_into_local_tvf.out | 2 +
.../data/nereids_p0/test_cse_session_var_guard.out | 64 +
.../eliminate_gby_key/eliminate_group_by.out | 6 +-
.../org/apache/doris/regression/suite/Suite.groovy | 224 +-
.../regression/suite/SuiteMTMVTaskWaitTest.groovy | 76 +
.../test_sql_cache_over_arrow_flight.groovy | 15 +
.../auth_call/test_ddl_ai_resource_auth.groovy | 28 +-
.../use_vcg_read_write.groovy | 15 +-
.../iceberg/test_gen_iceberg_by_api.groovy | 9 +-
...erg_migrated_nested_without_name_mapping.groovy | 55 +
.../test_iceberg_schema_time_travel_matrix.groovy | 4 +-
.../test_iceberg_struct_schema_evolution.groovy | 9 +
.../test_iceberg_hadoop_catalog_kerberos.groovy | 68 +-
.../tvf/insert/test_insert_into_local_tvf.groovy | 36 +-
.../nereids_p0/test_cse_session_var_guard.groovy | 77 +
.../query_p0/cache/sql_cache_object_type.groovy | 7 +-
155 files changed, 15766 insertions(+), 6201 deletions(-)
create mode 100644 be/test/exec/runtime_filter/sync_size_callback_test.cpp
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/preinstalled_data/iceberg/migrated_nested_without_name_mapping/data/data.parquet
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/preinstalled_data/iceberg/migrated_nested_without_name_mapping/metadata/00000-4c8aafe0-9de1-4b3f-942d-305dc33fb82f.metadata.json
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/preinstalled_data/iceberg/migrated_nested_without_name_mapping/metadata/00001-94d09fba-61c6-4e4b-b0f5-6023f70d5033.metadata.json
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/preinstalled_data/iceberg/migrated_nested_without_name_mapping/metadata/00002-7493fffe-54f9-486a-b15e-7333fdb96f61.metadata.json
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/preinstalled_data/iceberg/migrated_nested_without_name_mapping/metadata/00003-97f3e871-e421-48ed-a735-354f4e1c9502.metadata.json
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/preinstalled_data/iceberg/migrated_nested_without_name_mapping/metadata/snap-2748466134835934544-1-94f693c5-6efb-4e9d-a40e-f02ac301fc14.avro
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/preinstalled_data/iceberg/migrated_nested_without_name_mapping/metadata/stage-7-task-401-manifest-d04009d1-83e2-4d2a-859e-79d57e8dd381.avro
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexDatasetLocator.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexFenceKey.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexJob.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexJobCompletionReason.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexJobManager.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexJobMutationState.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexJobMutationType.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexJobQuota.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexJobRefreshState.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexJobResult.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexJobResultCode.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexNameNormalizer.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexSchemaContract.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/job/LanceIndexTerminationProof.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/lance/job/LanceIndexDatasetLocatorTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/lance/job/LanceIndexJobManagerPersistTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/lance/job/LanceIndexJobManagerReplayTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/lance/job/LanceIndexJobQuotaTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/lance/job/LanceIndexJobResultClassifyTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/lance/job/LanceIndexJobStateMachineTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/lance/job/LanceIndexJobTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/lance/job/LanceIndexJobWiringTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/lance/job/LanceIndexNameNormalizerTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertTaskAuditTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/load/GroupCommitManagerTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/BindTVFTableSinkTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/EliminateGroupByAvgWeightedTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/AIResourceCommandTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/ExecuteActionCommandTest.java
create mode 100644
regression-test/data/external_table_p0/iceberg/test_iceberg_migrated_nested_without_name_mapping.out
create mode 100644
regression-test/data/nereids_p0/test_cse_session_var_guard.out
create mode 100644
regression-test/framework/src/test/groovy/org/apache/doris/regression/suite/SuiteMTMVTaskWaitTest.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_migrated_nested_without_name_mapping.groovy
create mode 100644
regression-test/suites/nereids_p0/test_cse_session_var_guard.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
