This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-catalog-spi
in repository https://gitbox.apache.org/repos/asf/doris.git
omit eb0921d56a2 [fix](catalog) port #65676 iceberg deletion-vector
metadata validation to the connector scan path
omit 0cebd657187 [fix](catalog) port #65548 external COUNT(*)/COUNT(col)
semantics to the connector scan path
omit c5aca7ed934 [fix](catalog) iceberg system-table scan: restore #65262
positional JNI read + order-preserving projection
omit ebf1846a970 [fix](catalog) restore jfs hive-write hadoop config +
iceberg system-table scan projection
omit a4858fb7bad [feat](catalog) fe-connector-iceberg: port #64966 REST 401
re-auth to the connector
omit 0cf940a36e8 [refactor](catalog) drop
ReauthenticatingRestSessionCatalog test orphaned by master rebase
omit e0bcd0633ee [refactor](catalog) fe-core: remove 6 dead datasource
classes orphaned by the SPI migration (#65740)
omit b7409f5ca9d [refactor](catalog) fe-core: remove dead
ExternalMetadataOps layer, deprecated LakeSoul catalog, and odbc/statistics
stubs (#65736)
omit 61ab7a11dcf [refactor](catalog) fe-core: remove dead Trino-ported
DirectoryListercache cluster (#65734)
omit 2eacfd426a3 [refactor](catalog) fe-connector: replace fat
hive-catalog-shade with a slim HMS metastore-client shade (#65721)
omit 6229e17e64c [refactor](catalog) Catalog spi 11 hive (#65473)
omit 447abcc1a51 [doc](catalog) HANDOFF: $position_deletes port done; e2e
is the only remaining flip gate
omit 8c2caf26da2 [feat](catalog) P6.6: support the iceberg
$position_deletes system table on the connector SPI
omit 3d58a1cc6af [doc](catalog) HANDOFF: point next step at the
$position_deletes port design (T0 evidence in)
omit 2ab2297d8ac [doc](catalog) P6.6 $position_deletes connector port:
research notes + design (T0 evidence in, ready to implement)
omit aa97017f6fe [doc](catalog) HANDOFF: rebase to upstream 30a44d61a8a +
register $position_deletes connector gap as flip blocker
omit d772fc57b5a [fix](build) fix parallel -T shade race in
fe-connector-paimon-hive-shade
omit 9203e048913 [fix](catalog) reconcile upstream #65365 with connector
SPI after rebase to 4f48ccfaa7d
omit 88edc39ad68 [doc](catalog) P6.6-C6 #63068 HANDOFF: record
cache-leakage data-flow test + Task 0 retained-base verify
omit 4367f526c97 [feat](catalog) P6.6-C6 #63068 FE: re-migrate
ExternalDatabaseSessionContextTest (cross-user cache-leakage data-flow)
omit e2045845454 [doc](catalog) P6.6-C6 #63068 OIDC session migration:
design + research notes + HANDOFF (Tasks 0-7 DONE)
omit a0c920b5ee0 [feat](catalog) P6.6-C6 #63068 iceberg: per-user REST
session routing (metadata + scan/write/procedure)
omit 34f8e8e5850 [feat](catalog) P6.6-C6 #63068 FE: capability-gated
credential injection + per-user cache bypass
omit 696c29b713b [feat](catalog) P6.6-C6 #63068 SPI: neutral
delegated-credential DTO + user-session capability
omit 2ea7c93db13 [fix](catalog) tier3 #65094: preserve external-table
top-level column-name case on the read path
omit 3927b4496bd [fix](catalog) port upstream #65332/#65094 datasource
fixes into SPI connectors
omit baec46b172f [fix](catalog) P4 maxcompute: drop dangling legacy
MaxComputeExternalTableTest after rebase
omit 9b1ea31ddd9 [fix](catalog) jdbc: keep driver classloaders alive per
URL to stop Metaspace leak
omit c23f9fcd088 [refactor](catalog) P6 iceberg: migrate to catalog SPI +
cutover + remove legacy fe-core subsystem (#64688)
omit 93737971747 [refactor](catalog) P3b: consolidate kerberos
authentication into fe-kerberos (#64655)
omit 5f168f3cb5a [refactor](catalog) P5 paimon: remove legacy subsystem
from fe-core + make fe-core paimon-SDK-free (T29) (#64653)
omit 1de7c82d38f [refactor](catalog) P5 paimon: migrate to catalog SPI +
cutover (#64446) (#64446)
omit 097d8780c6d update P5 handoff and fix compile issue
omit fdaf1631cc4 [refactor](connector) P4 maxcompute: remove legacy
subsystem from fe-core + make fe-core odps-free (T07-T09) (#64300)
omit 526e5dc3fb2 [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
omit 7115bdcdd65 [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
omit a2c82578337 [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
omit 8983df1a84c [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
omit b5cb4514c5b [doc](connector) add project tracking system for catalog
SPI migration
add 2c697190ccd [fix](be) Correct Arrow timestamps before Unix epoch in
DATETIMEV2 deserialization (#65689)
add 8b081ea92ac [fix](cloud) set recycler S3 client requestTimeoutMs to
avoid curl-28 on slow DeleteObjects (mirror #49315) (#64758)
add 52e154530ff [improvement](regression) Reduce Kerberos test environment
memory (#65564)
add 6322a095ba7 [fix](be) Materialize const columns before block merge
(#65770)
add c83166309e4 [fix](fe) Keep CDC end offset consistent with current
progress (#65688)
add 84b62c7e83b [doc](connector) add project tracking system for catalog
SPI migration
add 52d70c63344 [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
add edcc3dc3332 [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
add c8205e5f3f6 [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
add e92ed372e91 [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
add 4c4904ec86f [refactor](connector) P4 maxcompute: remove legacy
subsystem from fe-core + make fe-core odps-free (T07-T09) (#64300)
add 0dfda9eb416 update P5 handoff and fix compile issue
add fd59353f09c [refactor](catalog) P5 paimon: migrate to catalog SPI +
cutover (#64446) (#64446)
add a0aabc37638 [refactor](catalog) P5 paimon: remove legacy subsystem
from fe-core + make fe-core paimon-SDK-free (T29) (#64653)
add 57128e6210e [refactor](catalog) P3b: consolidate kerberos
authentication into fe-kerberos (#64655)
add 750bf1018c5 [refactor](catalog) P6 iceberg: migrate to catalog SPI +
cutover + remove legacy fe-core subsystem (#64688)
add c28a4fea547 [fix](catalog) jdbc: keep driver classloaders alive per
URL to stop Metaspace leak
add b3821d28a30 [fix](catalog) P4 maxcompute: drop dangling legacy
MaxComputeExternalTableTest after rebase
add 7f8c4dc9e02 [fix](catalog) port upstream #65332/#65094 datasource
fixes into SPI connectors
add 897723de723 [fix](catalog) tier3 #65094: preserve external-table
top-level column-name case on the read path
add cf703220183 [feat](catalog) P6.6-C6 #63068 SPI: neutral
delegated-credential DTO + user-session capability
add 7637ef4046e [feat](catalog) P6.6-C6 #63068 FE: capability-gated
credential injection + per-user cache bypass
add 1d4c6505777 [feat](catalog) P6.6-C6 #63068 iceberg: per-user REST
session routing (metadata + scan/write/procedure)
add 061cabdcad9 [doc](catalog) P6.6-C6 #63068 OIDC session migration:
design + research notes + HANDOFF (Tasks 0-7 DONE)
add 58a829b7c92 [feat](catalog) P6.6-C6 #63068 FE: re-migrate
ExternalDatabaseSessionContextTest (cross-user cache-leakage data-flow)
add 1e7b3063ea5 [doc](catalog) P6.6-C6 #63068 HANDOFF: record
cache-leakage data-flow test + Task 0 retained-base verify
add 674eb26f91c [fix](catalog) reconcile upstream #65365 with connector
SPI after rebase to 4f48ccfaa7d
add a25f2f0b9a2 [fix](build) fix parallel -T shade race in
fe-connector-paimon-hive-shade
add 01c09fc789b [doc](catalog) HANDOFF: rebase to upstream 30a44d61a8a +
register $position_deletes connector gap as flip blocker
add 798dc7b7d67 [doc](catalog) P6.6 $position_deletes connector port:
research notes + design (T0 evidence in, ready to implement)
add 3018112ca67 [doc](catalog) HANDOFF: point next step at the
$position_deletes port design (T0 evidence in)
add 4737f9485cc [feat](catalog) P6.6: support the iceberg
$position_deletes system table on the connector SPI
add b5572c26cce [doc](catalog) HANDOFF: $position_deletes port done; e2e
is the only remaining flip gate
add 661211c7f0a [refactor](catalog) Catalog spi 11 hive (#65473)
add 0694b7fb78c [refactor](catalog) fe-connector: replace fat
hive-catalog-shade with a slim HMS metastore-client shade (#65721)
add fdda9928014 [refactor](catalog) fe-core: remove dead Trino-ported
DirectoryListercache cluster (#65734)
add 5990b8400bf [refactor](catalog) fe-core: remove dead
ExternalMetadataOps layer, deprecated LakeSoul catalog, and odbc/statistics
stubs (#65736)
add 6ccae1ff9c7 [refactor](catalog) fe-core: remove 6 dead datasource
classes orphaned by the SPI migration (#65740)
add e1dba2a75fc [refactor](catalog) drop
ReauthenticatingRestSessionCatalog test orphaned by master rebase
add 034cec09f38 [feat](catalog) fe-connector-iceberg: port #64966 REST 401
re-auth to the connector
add d4231be7914 [fix](catalog) restore jfs hive-write hadoop config +
iceberg system-table scan projection
add 6e3191abd1f [fix](catalog) iceberg system-table scan: restore #65262
positional JNI read + order-preserving projection
add f2ec03489b9 [fix](catalog) port #65548 external COUNT(*)/COUNT(col)
semantics to the connector scan path
add 442f23dc731 [fix](catalog) port #65676 iceberg deletion-vector
metadata validation to the connector scan path
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (eb0921d56a2)
\
N -- N -- N refs/heads/branch-catalog-spi (442f23dc731)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/core/block/block.h | 11 ++
.../data_type_serde/data_type_datetimev2_serde.cpp | 20 ++-
be/test/core/block/block_test.cpp | 31 ++++
.../data_type_serde_datetime_v2_test.cpp | 29 ++++
cloud/src/recycler/s3_accessor.cpp | 5 +
.../docker-compose/kerberos/.gitignore | 8 +
.../docker-compose/kerberos/Dockerfile | 24 +--
.../kerberos/common/conf/doris-krb5.conf | 6 -
.../common/hadoop/apply-config-overrides.sh | 31 ----
.../kerberos/common/hadoop/hadoop-run.sh | 44 -----
.../auth-to-local.xml => conf/core-site.xml.tpl} | 29 +++-
.../docker-compose/kerberos/conf/hdfs-site.xml.tpl | 113 ++++++++++++
.../docker-compose/kerberos/conf/hive-site.xml.tpl | 65 +++++++
.../docker-compose/kerberos/conf/kdc.conf.tpl | 18 +-
.../kerberos/conf/kerberos1/kdc.conf.tpl | 50 ------
.../kerberos/conf/kerberos1/krb5.conf.tpl | 39 -----
.../kerberos/conf/kerberos2/kdc.conf.tpl | 38 ----
.../kerberos/conf/{kerberos2 => }/krb5.conf.tpl | 16 +-
.../docker-compose/kerberos/conf/my.cnf.tpl | 38 ----
.../kerberos/entrypoint-hive-master.sh | 192 +++++++++++++--------
.../docker-compose/kerberos/hadoop-hive.env.tpl | 120 +++----------
.../kerberos/health-checks/health.sh | 22 +--
.../kerberos/health-checks/hive-health-check-2.sh | 20 ---
.../kerberos/health-checks/hive-health-check.sh | 20 ---
.../kerberos/health-checks/supervisorctl-check.sh | 40 -----
.../docker-compose/kerberos/kerberos.yaml.tpl | 68 +++-----
.../docker-compose/kerberos/kerberos1_settings.env | 49 +-----
.../docker-compose/kerberos/kerberos2_settings.env | 45 +----
.../kerberos/sql/create_kerberos_hive_table.sql | 17 --
.../kerberos/sql/create_paimon_hive_table.hql | 7 -
.../two-kerberos-hives/hive2-default-fs-site.xml | 25 ---
docker/thirdparties/run-thirdparties-docker.sh | 148 ++++++++++++++--
.../job/offset/jdbc/JdbcSourceOffsetProvider.java | 58 +++++--
.../offset/jdbc/JdbcTvfSourceOffsetProvider.java | 15 +-
.../jdbc/JdbcSourceOffsetProviderOffsetTest.java | 139 +++++++++++++++
.../spark_connector/spark_connector_arrow.out | 15 +-
.../test_information_schema_timezone.groovy | 3 +
.../test_iceberg_hadoop_catalog_kerberos.groovy | 6 +-
.../kerberos/test_non_catalog_kerberos.groovy | 19 +-
.../kerberos/test_single_hive_kerberos.groovy | 99 ++++++-----
.../kerberos/test_two_hive_kerberos.groovy | 34 ++++
.../refactor_storage_param/hdfs_all_test.groovy | 5 +-
.../paimon/test_paimon_hms_catalog.groovy | 7 +-
.../hive_on_hms_and_dlf.groovy | 14 +-
.../iceberg_on_hms_and_filesystem_and_dlf.groovy | 21 +--
...ob_cdc_stream_postgres_latest_alter_cred.groovy | 11 +-
46 files changed, 1000 insertions(+), 834 deletions(-)
create mode 100644 docker/thirdparties/docker-compose/kerberos/.gitignore
copy be/src/storage/index/ann/CMakeLists.txt =>
docker/thirdparties/docker-compose/kerberos/Dockerfile (61%)
delete mode 100755
docker/thirdparties/docker-compose/kerberos/common/hadoop/apply-config-overrides.sh
delete mode 100755
docker/thirdparties/docker-compose/kerberos/common/hadoop/hadoop-run.sh
rename
docker/thirdparties/docker-compose/kerberos/{two-kerberos-hives/auth-to-local.xml
=> conf/core-site.xml.tpl} (58%)
mode change 100755 => 100644
create mode 100644
docker/thirdparties/docker-compose/kerberos/conf/hdfs-site.xml.tpl
create mode 100644
docker/thirdparties/docker-compose/kerberos/conf/hive-site.xml.tpl
copy .clangd => docker/thirdparties/docker-compose/kerberos/conf/kdc.conf.tpl
(74%)
delete mode 100644
docker/thirdparties/docker-compose/kerberos/conf/kerberos1/kdc.conf.tpl
delete mode 100644
docker/thirdparties/docker-compose/kerberos/conf/kerberos1/krb5.conf.tpl
delete mode 100644
docker/thirdparties/docker-compose/kerberos/conf/kerberos2/kdc.conf.tpl
rename docker/thirdparties/docker-compose/kerberos/conf/{kerberos2 =>
}/krb5.conf.tpl (81%)
delete mode 100644 docker/thirdparties/docker-compose/kerberos/conf/my.cnf.tpl
delete mode 100755
docker/thirdparties/docker-compose/kerberos/health-checks/hive-health-check-2.sh
delete mode 100755
docker/thirdparties/docker-compose/kerberos/health-checks/hive-health-check.sh
delete mode 100755
docker/thirdparties/docker-compose/kerberos/health-checks/supervisorctl-check.sh
delete mode 100644
docker/thirdparties/docker-compose/kerberos/sql/create_kerberos_hive_table.sql
delete mode 100644
docker/thirdparties/docker-compose/kerberos/sql/create_paimon_hive_table.hql
delete mode 100755
docker/thirdparties/docker-compose/kerberos/two-kerberos-hives/hive2-default-fs-site.xml
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/job/offset/jdbc/JdbcSourceOffsetProviderOffsetTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
