This is an automated email from the ASF dual-hosted git repository.
voonhous pushed a change to branch release-1.2.1
in repository https://gitbox.apache.org/repos/asf/hudi.git
from 23d50905d6e8 test(trino): de-flake TestHudi*FileOperations by
disabling async table statistics (#18995)
new 709767094682 perf(metadata): Avoid per-record enum-array clone and
string parse when materializing MDT records (#18997)
new 4159a94a3e88 [MINOR] Cap UT_FT_10 Azure install to -T 2 to avoid flaky
compiler heap OOM (#19008)
new e2af6db031d9 refactor(metadata): Replace misused stream reduce with a
plain for-loop (#18532)
new d994e06674d5 perf(io): Derive log file size from AppendResult on
append-handle close (#19002)
new b38994a22b17 refactor: Add Lombok annotations to hudi-common module
(part 8) (#18957)
new c2681b229b32 [MINOR] Wait for ZK connection in lock provider to
de-flake direct-marker detection test (#19014)
new 969a7e8ef050 fix(flink): fix the mor small file record size estimation
(#18991)
new a6bf0f608808 perf(kafka-connect): reuse AvroConvertor across records
in the connect writer (#19015)
new d53340c37759 fix(spark): Propagate merge configs to file group reader
during clustering (#19007)
new 4f4229f7de85 perf(kafka-connect): memoize file id per partition path
in the connect writer (#19016)
new 98ad512fe59e perf(kafka-connect): use a pre-sized ArrayList when
flushing buffered records (#19017)
new 71cef99d6e29 chore(test): document macOS-specific consistency-guard
slowness in cleaner test (#17714) (#19003)
new 5d81ce44b269 perf(metadata): Resolve column-stats field schemas once
per collection instead of per record (#19000)
new 8e08db363317 test(hadoop-mr): enable rollback case in
HoodieRealtimeRecordReader.testReader (#18693)
new 9b701ca1e27e feat(flink): Support writing out-of-line BLOB columns
(#18958)
The 15 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
azure-pipelines-20230430.yml | 8 +-
.../lock/BaseZookeeperBasedLockProvider.java | 27 ++-
.../org/apache/hudi/io/HoodieAppendHandle.java | 16 +-
.../strategy/ClusteringExecutionStrategy.java | 6 +-
.../TestZookeeperBasedLockProvider.java | 21 ++
.../apache/hudi/util/HoodieSchemaConverter.java | 44 ++--
.../apache/hudi/util/RowDataToAvroConverters.java | 8 +
.../hudi/util/TestHoodieSchemaConverter.java | 17 ++
...erBasedDetectionStrategyWithZKLockProvider.java | 29 ++-
.../hudi/common/table/log/TestLogReaderUtils.java | 58 ++++++
.../clean/TestCleanerInsertAndCleanByVersions.java | 4 +
.../common/table/timeline/BaseHoodieTimeline.java | 13 +-
.../hudi/common/table/timeline/HoodieInstant.java | 33 +--
.../hudi/common/table/timeline/LSMTimeline.java | 11 +-
.../table/timeline/MetadataConversionUtils.java | 11 +-
.../common/table/timeline/TimeGeneratorBase.java | 12 +-
.../common/table/timeline/TimelineDiffHelper.java | 46 ++---
.../hudi/common/table/timeline/TimelineLayout.java | 58 +-----
.../hudi/common/table/timeline/TimelineUtils.java | 13 +-
.../timeline/versioning/TimelineLayoutVersion.java | 8 +-
.../timeline/versioning/v1/ActiveTimelineV1.java | 45 ++--
.../versioning/v1/ArchivedTimelineLoaderV1.java | 10 +-
.../timeline/versioning/v1/ArchivedTimelineV1.java | 11 +-
.../versioning/v1/CommitMetadataSerDeV1.java | 5 +-
.../versioning/v1/CompletionTimeQueryViewV1.java | 9 +-
.../timeline/versioning/v2/ActiveTimelineV2.java | 48 ++---
.../versioning/v2/CompletionTimeQueryViewV2.java | 7 +-
.../table/view/AbstractTableFileSystemView.java | 52 ++---
.../common/table/view/FileSystemViewManager.java | 27 ++-
.../table/view/HoodieTableFileSystemView.java | 15 +-
.../IncrementalTimelineSyncFileSystemView.java | 56 +++--
.../table/view/PriorityBasedFileSystemView.java | 25 +--
.../view/RemoteHoodieTableFileSystemView.java | 6 +-
.../table/view/RocksDbBasedFileSystemView.java | 50 +++--
.../view/SpillableMapBasedFileSystemView.java | 8 +-
.../hudi/metadata/HoodieTableMetadataUtil.java | 60 +++---
.../hudi/metadata/MetadataPartitionType.java | 20 +-
.../hudi/metadata/TestHoodieTableMetadataUtil.java | 22 ++
.../partitioner/profile/DeltaWriteProfile.java | 36 +++-
.../sink/partitioner/profile/WriteProfile.java | 60 +++---
.../hudi/sink/partitioner/TestBucketAssigner.java | 105 +++++++++-
.../org/apache/hudi/table/ITTestBlobWrite.java | 226 +++++++++++++++++++++
.../hudi/table/ITTestDynamicBucketStreamWrite.java | 7 +-
.../hudi/utils/TestRowDataToAvroConverters.java | 120 +++++++++++
.../realtime/TestHoodieRealtimeRecordReader.java | 13 +-
.../connect/writers/AbstractConnectWriter.java | 25 ++-
.../connect/writers/BufferedConnectWriter.java | 5 +-
.../procedure/TestClusteringWithCustomMerger.scala | 133 ++++++++++++
48 files changed, 1174 insertions(+), 475 deletions(-)
create mode 100644
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestBlobWrite.java
create mode 100644
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/procedure/TestClusteringWithCustomMerger.scala
