This is an automated email from the ASF dual-hosted git repository.
voonhous pushed a change to branch release-1.2.1
in repository https://gitbox.apache.org/repos/asf/hudi.git
discard 672c7bda884b Revert "fix(reader): Remove redundant partition value
conversion in RecordContext (#19201)"
discard 91f95a90fb11 Revert "test(spark): drop the adapter-dependent String
case from convertValueToEngineType"
discard fc942ea18b6b test(flink): mock FlinkAppendHandle for the MOR routing
assertions
discard 30b8ec6b89cf fix(flink): preserve Avro fixed decimal widths in Parquet
writes (#19522)
discard 99f6f3802713 fix(trino): port the connector to this branch's package
layout and metadata config
discard dfafea9c6238 test(metadata): drop the file-slice half of
wrapsMetadataReaderAndFileSliceReadFailures
discard 3bd76ead3e3f perf(flink): use a shared work-stealing split pool for
Source V2 bounded reads (#19520)
discard 47241bf647d7 test(spark): drop the adapter-dependent String case from
convertValueToEngineType
discard 091f02668830 fix(build): point the Trino module poms at this branch's
version
discard 7cbb3a259784 fix(common): throw IllegalStateException from the
checkState message-supplier overload
discard 859b73fa1a65 fix(client): report completed timeline action in
clustering callback (#19464)
discard 3e6f8f8123d4 fix(ci): drop the changes-gate residue that kept Java CI
from starting
discard 374636b1268d fix(build): import schema internal types from their
release-branch package
discard 684a6fa35203 fix(metrics): route the reflection failures a CloudWatch
skew actually produces (#19477)
discard 78b5f3f6784a fix(spark): preserve the Avro fixed-size decimal width in
the Spark row write support (#19512)
discard 245dd8851ba4 fix(flink): close lookup reader after cache reload
attempts (#19503)
discard 79e387aaf66d test(trino): add a Trino E2E testcontainers pipeline for
the RFC-105 connector (#19217)
discard b6a4eafe6d87 fix(metrics): do not drop the whole CloudWatch batch on
one unmappable metric name (#19476)
discard adffdda13fa2 fix(build): import Lazy from its release-branch package
in HoodieHadoopStorage
discard f4594d8058d5 fix(flink): close CDC image spillable maps on failures
(#19482)
discard 010397bf4221 perf(trino): drop the decimal schema cache and memoize
prefilled values (#19495)
discard a4828674a63a fix(trino): report real block size and slice splits
solely by target_split_size (#19478)
discard f27000078651 fix(schema): require a per-field override to promote a
bare long to a timestamp logical type (#19384)
discard b8a09bf2ef9d fix(fs): stop depending on the optional
FileSystem#getScheme() (#19470)
discard 58b114ad5c87 fix(meta-sync): advance last commit time synced when it
trails the active timeline midpoint (#19239)
discard 13c02c1bc81c perf(trino): cache decimal Avro schema in
HudiAvroSerializer instead … (#19483)
discard 1eaca70549eb fix(timeline): do not NPE on archived instants without a
completion time (#19452)
discard 0d31a40a19c6 fix(metrics): explain how to enable the CloudWatch
reporter when hudi-aws is absent (#19418)
discard 91d56de76a55 test(java-client): cover both marker types with the
embedded timeline server (#19444)
discard f7155222a298 fix(trino): remap pushed-down predicate columns to
physical file ordinals (#19456)
discard d8dac86db9f6 fix(flink): rethrow StreamWriteOperatorCoordinator
start() failures (#19432)
discard 134963765ed4 fix(trino): read uncompacted MDT HFILE log deltas and
guard index pruning (#19298)
discard a6884f975372 fix(utilities): include metadata table index-init
instants in the record index validation snapshot (#19395)
discard 2f5878549351 fix(hive-sync): make skip_ro_suffix take precedence over
sync_snapshot_with_table_name (#19427)
discard 494359e00847 perf(common): avoid UTF-8 allocations in string
comparator (#19414)
discard 469b8c1cb538 feat(client): enrich write commit callback message and
fire it for table-service commits (#18988)
discard 5050c2663679 test(trino): add MoR read tests for delete markers,
custom payloads and commit-time ordering (#19295)
discard 4db6e16aa62b feat(release): add a JDK 25 staging path for
org.apache.hudi:hudi-trino (#19410)
discard 68fd7cb4caa4 test(flink): improve streamer config and schema provider
coverage (#19392)
discard 856776effef1 fix(client): fix NPE in schema conflict resolution on
commits with null writer schema (#19388)
discard d7fbe3234f82 feat(trino): resolve merge-required columns from the
table schema (#19288)
discard 8a435ac951b5 fix(hive-sync): stop HiveDriverPool from swallowing a
failed SQL batch (#19391)
discard a371f8b95f61 fix(reader): give the file group reader schema handler
the merged record-merge properties (#19389)
discard 538e74649433 fix(build): drop duplicate MetricRegistry import in
TestHoodieIncrSource
discard d066e9bc3431 fix(schema): gate timestamp-precision change behind a
per-field verdict (#19029)
discard d12eb478849e fix(common): load single archived instant details (#19385)
discard f624d8781f76 fix(metadata): correct index definition lookup and
improve mdt read coverage (#19359)
discard 391324bdea73 fix(hive-sync): drop the unreachable HMS lock
timeout-recovery path (#19371)
discard 2a584ca8b8de feat(hive-sync): batch and parallelize HiveQL partition
operations (#18984)
discard cdc9c565208f feat(trino): Migrate the Trino-Hudi connector into the
Hudi repo (RFC-105) (#18837)
discard 3d4edab74f49 fix: Fix lock expiration metric (#18492)
discard 3730b6015405 fix(build): use log4j-slf4j2-impl in hudi-flink1.17.x
discard 591401fc3e46 perf(flink): Parse bucket index hash-field config once
instead of per record (#18993)
discard 5dbbe50dd481 fix(record-index-bootstrap): sort record index keys by
UTF-8 bytes to match HFile sorting (#18941)
discard 7f843e6d786c fix(hive-sync): stop the HMS lock heartbeat once the
metastore drops the lock (#19370)
discard 212ede61384a chore(deps): bump
org.apache.commons:commons-configuration2 (#18801)
discard 9913db7ed8e8 chore(deps): bump org.apache.thrift:libthrift (#18910)
discard 06f3c925c88a chore(deps): bump io.airlift:aircompressor from 0.27 to
2.0.3 (#18247)
discard 944e86d39bd1 test(hudi-client): improve metadata table writer coverage
(#19363)
discard 1bc5d3869653 fix(timeline-service): fail marker creation requests when
marker flush fails (#19368)
discard f7eecba4bd0d fix(flink): deduplicate delete keys and cover write paths
(#19354)
discard 78f27a475cec fix(flink): normalize row logical conversions and improve
coverage (#19351)
discard 97279c5de6c9 fix(metadata-table): follow-ups for the zero-size file
skip (#18611) (#19355)
discard fd231034e166 docs(docker): document build_docker_images.sh flags
(#18687)
discard b963291b9fe9 test(spark): Fix flaky TestSparkFilterHelper by giving it
its own SparkSession (#19356)
discard 6fa05d5cb20e fix: Handle map/array-nested leaf columns in column stats
collection during MOR log-append (#19126)
discard 7e46473c1758 fix(flink): make hudi-flink-bundle built with
flink-bundle-shade-hive usable for Hive sync (#19330)
discard 16f6573669be test(common): add LSM file group read path coverage
(#19347)
discard 85e49e73b842 fix(reader): derive pre-v9 CDC delete markers from the
effective payload class (#19348)
discard 6bec73452958 chore(utilities): add SQS backlog / in-flight visibility
to S3 events source (#19333)
discard 93cb853668de fix(utilities): guard Source.releaseResources() against
transient RDD unpersist failures (#19328)
discard c026a7706b1b fix(hive-sync): keep HMS lock heartbeat alive and release
its thread pool on close (#19334)
discard c48eb02cfd81 [HUDI-18060] Improve error message when ordering field
value is null (#18061)
discard 4d143a884fb0 fix(build): move TestSparkValueMetadataUtils to the
release-branch stats package
discard 70982d8d0dbd refactor(spark): share the Spark 3.x legacy parquet file
format via a common base (#19341)
discard 82c957fa1a9e fix: do not fall back to timeline server markers on
transient HDFS failures (#18887)
discard 2147325bc30c chore: upload Flink integration-test coverage (#19343)
discard d175f92a41d4 fix(hive-sync): set HMS table createTime in seconds
instead of milliseconds (#19335)
discard 6d50d8289201 fix: relax existing column to nullable in reconcileSchema
when source made it nullable (#19337)
discard 9b99311f94fa fix(hive-sync): fix partition-value parsing on '=' and
pushdown comparator overflow (#19336)
discard d0b2a88020a0 test(common): add unit coverage for metrics reporters and
schema utilities (#19221)
discard c57db60266be fix(build): import HoodieStorageUtils from its
release-branch package
discard 5e17c57b282c fix(flink): avoid reusing split reader functions across
fetchers (#19315)
discard cdf8e9e768e6 fix(hive-sync): close proxied IMetaStoreClient in
HoodieHiveSyncClient.close() to prevent HMS connection leak (#19331)
discard 688c5c24448d test(client): add unit coverage for client utilities and
services (#19222)
discard 51f693c0cb4e test(common): add unit coverage for config and utility
classes (#19220)
discard 1eefd1b0a274 test(spark): add unit coverage for Spark row, schema and
sort utilities (#19219)
discard b653a5edc5e7 fix(hive-sync): sync column and partition column comments
to HMS (#19289)
discard f0f9598d57f5 fix(build): use CURRENT_VERSION for the inline log-format
header on release-1.2.1
discard a9d2e8140743 fix(metadata-table): add config to skip zero-size data
files on MDT initialization (#18611)
discard 9e65e525e906 test(common): add log-format reader and scanner coverage
(#19223)
discard c200e71e2abd fix(test): stabilize flaky
testReattemptOfFailedClusteringCommit (#19120)
discard 63b7a3415e15 docs: claim RFC-108 Multi-dataset incremental reads in
Hudi Streamer (#19308)
discard 706f920da978 fix(flink): remove the Source V2 read teardown race via
materialized bounded minibatches (#19202)
discard dde78ad6200c fix: Improve error message for conflict resolution
(#18119)
discard 19674a3c2c21 fix: reuse Inflater/Deflater in BitCaskDiskMap to avoid
JDK8 finalizer contention (#18818)
discard 85550a2b6034 test(spark): cover the legacy parquet read path with
file-group reader disabled (#19133)
discard 1f52a3a78101 fix(spark): match the staged table, not LogicalWriteInfo,
in BasicStagedTable.newWriteBuilder (#19251)
discard 12d25eb2bef8 fix: align log4j2 and slf4j versions to resolve IntelliJ
test failures (#18177)
discard a7b2f889fa7a Remove spaces around partition columns (#18423)
discard e22eaadd06bf fix(build): import Spark Metadata/StructField types in
TestHoodieSparkLanceWriter
discard c19105f07e2d fix(build): import lock result enums from their
release-branch package
discard 5b3fe07bc716 fix(spark): read INLINE blobs as CONTENT on internal
write-side Lance… (#19236)
discard 46d3c8494aa2 fix(build): import OperationConverter from its
release-branch package
discard 96bb7b2ccc93 fix(flink): prevent data loss on global failover for
streaming writes (#19237)
discard 265768229f3c refactor(spark): consolidate the vendored 3.x Avro serde
forks into hudi-spark3-common (#19168)
discard 51a52547568a fix(common): name the offending expression in
BindVisitor's unsupported-predicate error (#19241)
discard fd3f51053b77 fix(spark): read Lance BLOB columns in <=512-row chunks
to avoid lance-core FFI abort (#19181)
discard dc5c845de223 test(spark): add extended SQL parser coverage for index
DDL and Hudi column types (#19218)
discard 61a3a496e5e6 fix(bundle): shade parquet-variant into common bundle
includes for Spark 4.1+ (#19235)
discard 71b90a97e900 test(integ-test): add Spark 4.1.1 stack to hive-sync E2E
matrix (#19216)
discard 86871e04030c test(client): cover low-coverage small classes across
hudi-client (#19224)
discard 82f519e5f5ea fix(spark): demote spurious per-write INFO logs to debug
(#19170)
discard adebc4c95682 test(integ-test): add Testcontainers E2E for
VECTOR/BLOB/VARIANT Hive… (#19203)
discard 72fee44b3e28 fix(reader): Remove redundant partition value conversion
in RecordContext (#19201)
add ac17e61c683e test(integ-test): add Testcontainers E2E for
VECTOR/BLOB/VARIANT Hive… (#19203)
add c0cde80f8f43 fix(spark): demote spurious per-write INFO logs to debug
(#19170)
add 53c2bc71394a test(client): cover low-coverage small classes across
hudi-client (#19224)
add 177f67de7ea9 test(integ-test): add Spark 4.1.1 stack to hive-sync E2E
matrix (#19216)
add ad88cb3e9109 fix(bundle): shade parquet-variant into common bundle
includes for Spark 4.1+ (#19235)
add 24bd11b2b4cf test(spark): add extended SQL parser coverage for index
DDL and Hudi column types (#19218)
add 61655026b978 fix(spark): read Lance BLOB columns in <=512-row chunks
to avoid lance-core FFI abort (#19181)
add a8d886845ce5 fix(common): name the offending expression in
BindVisitor's unsupported-predicate error (#19241)
add 51ebdf91257c refactor(spark): consolidate the vendored 3.x Avro serde
forks into hudi-spark3-common (#19168)
add 133eee39cf3c fix(flink): prevent data loss on global failover for
streaming writes (#19237)
add 4c1b2a4956e5 fix(build): import OperationConverter from its
release-branch package
add 68b134cfb35c fix(spark): read INLINE blobs as CONTENT on internal
write-side Lance… (#19236)
add 443685080964 fix(build): import lock result enums from their
release-branch package
add 9049cc59dbc4 fix(build): import Spark Metadata/StructField types in
TestHoodieSparkLanceWriter
add 04b5c0ec2fb5 Remove spaces around partition columns (#18423)
add 0068f21c085f fix: align log4j2 and slf4j versions to resolve IntelliJ
test failures (#18177)
add b57870f1a947 fix(spark): match the staged table, not LogicalWriteInfo,
in BasicStagedTable.newWriteBuilder (#19251)
add d1d385e8a18a test(spark): cover the legacy parquet read path with
file-group reader disabled (#19133)
add 2576cec42269 fix: reuse Inflater/Deflater in BitCaskDiskMap to avoid
JDK8 finalizer contention (#18818)
add 05957b865c0e fix: Improve error message for conflict resolution
(#18119)
add e6a28c8c1d21 fix(flink): remove the Source V2 read teardown race via
materialized bounded minibatches (#19202)
add 814c294fd3e8 docs: claim RFC-108 Multi-dataset incremental reads in
Hudi Streamer (#19308)
add f32a5fea1b46 fix(test): stabilize flaky
testReattemptOfFailedClusteringCommit (#19120)
add ff098567e39e test(common): add log-format reader and scanner coverage
(#19223)
add ab4a6d6f6c1a fix(metadata-table): add config to skip zero-size data
files on MDT initialization (#18611)
add 3760a3c8cb7a fix(build): use CURRENT_VERSION for the inline log-format
header on release-1.2.1
add d8d369990f17 fix(hive-sync): sync column and partition column comments
to HMS (#19289)
add 513d2736983e test(spark): add unit coverage for Spark row, schema and
sort utilities (#19219)
add eaa2af653ab7 test(common): add unit coverage for config and utility
classes (#19220)
add 7cdc03e8bd49 test(client): add unit coverage for client utilities and
services (#19222)
add 128e962abc57 fix(hive-sync): close proxied IMetaStoreClient in
HoodieHiveSyncClient.close() to prevent HMS connection leak (#19331)
add a3bdfcc52a74 fix(flink): avoid reusing split reader functions across
fetchers (#19315)
add 6ff2463de8de fix(build): import HoodieStorageUtils from its
release-branch package
add 0d279d70322c test(common): add unit coverage for metrics reporters and
schema utilities (#19221)
add 6e8d46d27634 fix(hive-sync): fix partition-value parsing on '=' and
pushdown comparator overflow (#19336)
add 46468ab15ff9 fix: relax existing column to nullable in reconcileSchema
when source made it nullable (#19337)
add 55a0957e0a95 fix(hive-sync): set HMS table createTime in seconds
instead of milliseconds (#19335)
add 9d4c6036179f chore: upload Flink integration-test coverage (#19343)
add 0311b3bb19f9 fix: do not fall back to timeline server markers on
transient HDFS failures (#18887)
add d08d653760cf refactor(spark): share the Spark 3.x legacy parquet file
format via a common base (#19341)
add 4236e7064ec3 fix(build): move TestSparkValueMetadataUtils to the
release-branch stats package
add d63a00d5f01e [HUDI-18060] Improve error message when ordering field
value is null (#18061)
add 4b6081999c70 fix(hive-sync): keep HMS lock heartbeat alive and release
its thread pool on close (#19334)
add ef9a7f347abd fix(utilities): guard Source.releaseResources() against
transient RDD unpersist failures (#19328)
add 696fe6c4c355 chore(utilities): add SQS backlog / in-flight visibility
to S3 events source (#19333)
add f69f68eab6d2 fix(reader): derive pre-v9 CDC delete markers from the
effective payload class (#19348)
add 028357b11b11 test(common): add LSM file group read path coverage
(#19347)
add b3756517b322 fix(flink): make hudi-flink-bundle built with
flink-bundle-shade-hive usable for Hive sync (#19330)
add 1c9fd002ba0a fix: Handle map/array-nested leaf columns in column stats
collection during MOR log-append (#19126)
add a03241d79118 test(spark): Fix flaky TestSparkFilterHelper by giving it
its own SparkSession (#19356)
add 500ccc964454 docs(docker): document build_docker_images.sh flags
(#18687)
add a82843160333 fix(metadata-table): follow-ups for the zero-size file
skip (#18611) (#19355)
add a51b3a481fb9 fix(flink): normalize row logical conversions and improve
coverage (#19351)
add 67b1072602bc fix(flink): deduplicate delete keys and cover write paths
(#19354)
add 349bbe038498 fix(timeline-service): fail marker creation requests when
marker flush fails (#19368)
add a96c592a49c2 test(hudi-client): improve metadata table writer coverage
(#19363)
add 3c3a76db4805 chore(deps): bump io.airlift:aircompressor from 0.27 to
2.0.3 (#18247)
add 8199b6b798ef chore(deps): bump org.apache.thrift:libthrift (#18910)
add 278fc83d33ba chore(deps): bump
org.apache.commons:commons-configuration2 (#18801)
add 4ac33ddcaa74 fix(hive-sync): stop the HMS lock heartbeat once the
metastore drops the lock (#19370)
add 64aac8b974ea fix(record-index-bootstrap): sort record index keys by
UTF-8 bytes to match HFile sorting (#18941)
add ef9e70ea77d7 perf(flink): Parse bucket index hash-field config once
instead of per record (#18993)
add b5a36f7c5887 fix(build): use log4j-slf4j2-impl in hudi-flink1.17.x
add 36fc3a27b48e fix: Fix lock expiration metric (#18492)
add 74d1097335d4 feat(trino): Migrate the Trino-Hudi connector into the
Hudi repo (RFC-105) (#18837)
add db4710e3818b feat(hive-sync): batch and parallelize HiveQL partition
operations (#18984)
add 56b933050216 fix(hive-sync): drop the unreachable HMS lock
timeout-recovery path (#19371)
add 43fd161a02bc fix(metadata): correct index definition lookup and
improve mdt read coverage (#19359)
add 71072a60b7d3 fix(common): load single archived instant details (#19385)
add 1e02ba584237 fix(schema): gate timestamp-precision change behind a
per-field verdict (#19029)
add b3c7115524b6 fix(build): drop duplicate MetricRegistry import in
TestHoodieIncrSource
add 07da1c5ae465 fix(reader): give the file group reader schema handler
the merged record-merge properties (#19389)
add 63f32d8120c4 fix(hive-sync): stop HiveDriverPool from swallowing a
failed SQL batch (#19391)
add dd454cd2a92b feat(trino): resolve merge-required columns from the
table schema (#19288)
add b60d76f5b1a3 fix(client): fix NPE in schema conflict resolution on
commits with null writer schema (#19388)
add 7b1f070087bb test(flink): improve streamer config and schema provider
coverage (#19392)
add 1f92b1959647 feat(release): add a JDK 25 staging path for
org.apache.hudi:hudi-trino (#19410)
add 0ee5be70747b test(trino): add MoR read tests for delete markers,
custom payloads and commit-time ordering (#19295)
add 233bf0d12803 feat(client): enrich write commit callback message and
fire it for table-service commits (#18988)
add abea033d67a7 perf(common): avoid UTF-8 allocations in string
comparator (#19414)
add f96c7c1ac2d6 fix(hive-sync): make skip_ro_suffix take precedence over
sync_snapshot_with_table_name (#19427)
add 2599498dd364 fix(utilities): include metadata table index-init
instants in the record index validation snapshot (#19395)
add bc1923558f43 fix(trino): read uncompacted MDT HFILE log deltas and
guard index pruning (#19298)
add c31aa56e6a76 fix(flink): rethrow StreamWriteOperatorCoordinator
start() failures (#19432)
add 793f22c62819 fix(trino): remap pushed-down predicate columns to
physical file ordinals (#19456)
add 99c77c7382f8 test(java-client): cover both marker types with the
embedded timeline server (#19444)
add ad9682ca914d fix(metrics): explain how to enable the CloudWatch
reporter when hudi-aws is absent (#19418)
add 05591536405e fix(timeline): do not NPE on archived instants without a
completion time (#19452)
add 544d5f85793e perf(trino): cache decimal Avro schema in
HudiAvroSerializer instead … (#19483)
add b55a86a16191 fix(meta-sync): advance last commit time synced when it
trails the active timeline midpoint (#19239)
add 37f7d888b33b fix(fs): stop depending on the optional
FileSystem#getScheme() (#19470)
add 2df42dee8b4e fix(schema): require a per-field override to promote a
bare long to a timestamp logical type (#19384)
add f6832a5a1821 fix(trino): report real block size and slice splits
solely by target_split_size (#19478)
add 37cf01d265d8 perf(trino): drop the decimal schema cache and memoize
prefilled values (#19495)
add 392d76335c71 fix(flink): close CDC image spillable maps on failures
(#19482)
add 662fd5fe8344 fix(build): import Lazy from its release-branch package
in HoodieHadoopStorage
add ea9dda08aa0b fix(metrics): do not drop the whole CloudWatch batch on
one unmappable metric name (#19476)
add a85860f880ba test(trino): add a Trino E2E testcontainers pipeline for
the RFC-105 connector (#19217)
add aaec9112496d fix(flink): close lookup reader after cache reload
attempts (#19503)
add a7f7aaf212e6 fix(spark): preserve the Avro fixed-size decimal width in
the Spark row write support (#19512)
add c96f992a7295 fix(metrics): route the reflection failures a CloudWatch
skew actually produces (#19477)
add 339438feccf3 fix(build): import schema internal types from their
release-branch package
add 7a082ae62b44 fix(ci): drop the changes-gate residue that kept Java CI
from starting
add 3b8f0ec47595 fix(client): report completed timeline action in
clustering callback (#19464)
add 9fd9089902b1 fix(common): throw IllegalStateException from the
checkState message-supplier overload
add f4e3ed21661e fix(build): point the Trino module poms at this branch's
version
add 4ba73aa33da7 perf(flink): use a shared work-stealing split pool for
Source V2 bounded reads (#19520)
add a7ad6428f4e6 test(metadata): drop the file-slice half of
wrapsMetadataReaderAndFileSliceReadFailures
add 4079baf7063b fix(trino): port the connector to this branch's package
layout and metadata config
add e94717d6527d fix(flink): preserve Avro fixed decimal widths in Parquet
writes (#19522)
add 91a69695d25f test(flink): mock FlinkAppendHandle for the MOR routing
assertions
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (672c7bda884b)
\
N -- N -- N refs/heads/release-1.2.1 (91a69695d25f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
