This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a change to branch release-1.3
in repository https://gitbox.apache.org/repos/asf/paimon.git
from 2e11817b77 [spark] Fix group by partial partition of a multi partition
table (#6375)
new 673471f320 [spark] Improve the exception msg for unsupported type
(#6379)
new 2591ce20d5 [hotfix] Fix typo in MergeIntoPaimonTable (#6381)
new 867629ec23 [spark] Ensure compatibility of resolveFilter in lower
version spark3.4 (#6387)
new f325c44a43 [pvfs] Fix file status and input stream for PVFS (#6397)
new b1e7fd1287 [flink] support performing incremental clustering by flink
(#6395)
new 0d1a61ebeb [parquet] Bump parquet version to 1.15.2 (#6363)
The 6 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../content/append-table/incremental-clustering.md | 49 ++-
.../append/cluster/IncrementalClusterManager.java | 3 +-
.../apache/paimon/flink/action/CompactAction.java | 153 +++++++-
.../cluster/IncrementalClusterSplitSource.java | 118 ++++++
.../RemoveClusterBeforeFilesOperator.java} | 9 +-
...writeIncrementalClusterCommittableOperator.java | 112 ++++++
.../action/IncrementalClusterActionITCase.java | 413 +++++++++++++++++++++
paimon-format/src/main/resources/META-INF/NOTICE | 12 +-
.../analysis/expressions/ExpressionHelper.scala | 23 --
.../paimon/spark/procedure/CompactProcedure.java | 2 +-
.../analysis/expressions/ExpressionHelper.scala | 38 +-
.../spark/commands/MergeIntoPaimonTable.scala | 2 +-
.../spark/sql/InsertOverwriteTableTestBase.scala | 17 +-
.../paimon/spark/sql/MergeIntoTableTestBase.scala | 2 +-
.../apache/spark/sql/paimon/shims/Spark3Shim.scala | 3 +-
.../paimon/vfs/hadoop/PaimonVirtualFileSystem.java | 12 +-
.../apache/paimon/vfs/hadoop/VFSInputStream.java | 5 +
.../paimon/vfs/hadoop/VirtualFileSystemTest.java | 2 +
pom.xml | 2 +-
19 files changed, 900 insertions(+), 77 deletions(-)
create mode 100644
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/cluster/IncrementalClusterSplitSource.java
copy
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/{postpone/RemovePostponeBucketFilesOperator.java
=> cluster/RemoveClusterBeforeFilesOperator.java} (89%)
create mode 100644
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/cluster/RewriteIncrementalClusterCommittableOperator.java
create mode 100644
paimon-flink/paimon-flink-common/src/test/java/org/apache/paimon/flink/action/IncrementalClusterActionITCase.java
