This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a change to branch release-1.3
in repository https://gitbox.apache.org/repos/asf/paimon.git
from 26dfce04e3 [doc] Fix error link in views page
new 75b44862bb [core] Support non null column with write type (#6513)
new 3d09deb93d [hotfix] Add more informat to check partition spec in
InternalRowPartitionComputer
new 81e24ab09a [hotfix] Print partition spec and type when error in
InternalRowPartitionComputer
new f97750d196 [Python] Keep the variable names of Identifier consistent
with Java (#6520)
new 134e1b6976 [Python] Suppport multi prepare commit in the same
TableWrite (#6526)
new 47c521d3a7 [Python] Rename to BATCH_COMMIT_IDENTIFIER in snapshot.py
new e6ea490bf9 [python] support custom source split target size and split
open file cost (#6527)
new e233be189e [core] Fix that cannot get partition info if all files are
in level-0 when enable dv. (#6531)
new 048c0d4d83 [python] add test case for reading blob by to_iterator
(#6536)
new e9f4b2f3c3 [doc] fix postpone.default-bucket-num default value (#6554)
new e4ecfac05c [Python] Remove the use of reference types in
DATA_FILE_META_SCHEMA (#6549)
new 4b16ee2131 [common] Remove token expire time (#6544)
new 787210fbea [python] Fix AtomicType.to_dict() inconsistency with java
(#6548)
new ae57450853 [python] Fix File Source type in data file meta (#6571)
new ce4ebcdafe [Python] License added in test (#6572)
new a47a28ec38 [core] support incremental clustering in dv mode (#6559)
new 5360c0323f [core] Refactor deletion vectors support for incremental
cluster
new 1390842f9b [Python] Add Mixed read and write test between Java and
Python. (#6579)
The 18 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/paimon-python-checks.yml | 28 ++-
.../content/primary-key-table/data-distribution.md | 2 +-
.../org/apache/paimon/rest/RESTTokenFileIO.java | 2 -
.../paimon/utils/InternalRowPartitionComputer.java | 6 +-
.../append/cluster/HistoryPartitionCluster.java | 10 +-
.../append/cluster/IncrementalClusterManager.java | 120 ++++++++++--
.../org/apache/paimon/schema/SchemaValidation.java | 3 -
.../apache/paimon/table/sink/TableWriteImpl.java | 15 +-
.../paimon/table/system/PartitionsTable.java | 3 +-
.../test/java/org/apache/paimon/JavaPyE2ETest.java | 159 +++++++++++++++
.../cluster/IncrementalClusterManagerTest.java | 7 +-
.../paimon/table/DataEvolutionTableTest.java | 41 ++++
.../apache/paimon/flink/action/CompactAction.java | 26 ++-
.../cluster/IncrementalClusterSplitSource.java | 18 +-
.../cluster/RemoveClusterBeforeFilesOperator.java | 23 ++-
.../apache/paimon/flink/BatchFileStoreITCase.java | 10 +
.../action/IncrementalClusterActionITCase.java | 142 ++++++++++++++
paimon-python/dev/lint-python.sh | 49 ++++-
paimon-python/dev/run_mixed_tests.sh | 217 +++++++++++++++++++++
paimon-python/pypaimon/api/rest_api.py | 16 +-
paimon-python/pypaimon/common/core_options.py | 20 ++
paimon-python/pypaimon/common/identifier.py | 26 +--
paimon-python/pypaimon/common/memory_size.py | 201 +++++++++++++++++++
.../pypaimon/manifest/schema/data_file_meta.py | 10 +-
.../pypaimon/manifest/schema/manifest_file_meta.py | 4 +-
.../pypaimon/manifest/schema/simple_stats.py | 44 +++--
.../pypaimon/read/scanner/full_starting_scanner.py | 5 +-
paimon-python/pypaimon/schema/data_types.py | 8 +-
.../pypaimon/snapshot/catalog_snapshot_commit.py | 6 +-
paimon-python/pypaimon/snapshot/snapshot.py | 2 +
paimon-python/pypaimon/table/file_store_table.py | 5 +-
paimon-python/pypaimon/table/table.py | 6 +-
paimon-python/pypaimon/tests/blob_table_test.py | 80 ++++++++
paimon-python/pypaimon/tests/data_types_test.py | 67 +++++++
.../pypaimon/{api => tests/e2e}/__init__.py | 0
.../pypaimon/tests/e2e/java_py_read_write_test.py | 89 +++++++++
.../{catalog => tests/manifest}/__init__.py | 0
.../tests/manifest/manifest_schema_test.py | 151 ++++++++++++++
.../pypaimon/tests/py36/ao_simple_test.py | 56 ++++++
paimon-python/pypaimon/tests/reader_base_test.py | 132 +++++++++++++
paimon-python/pypaimon/tests/rest/rest_server.py | 18 +-
.../pypaimon/tests/write/table_write_test.py | 155 +++++++++++++++
paimon-python/pypaimon/write/file_store_commit.py | 2 +-
paimon-python/pypaimon/write/file_store_write.py | 5 +-
.../{batch_table_commit.py => table_commit.py} | 19 +-
.../write/{batch_table_write.py => table_write.py} | 29 ++-
.../{batch_write_builder.py => write_builder.py} | 35 +++-
.../pypaimon/write/writer/data_blob_writer.py | 2 +-
paimon-python/pypaimon/write/writer/data_writer.py | 2 +-
.../paimon/spark/procedure/CompactProcedure.java | 39 ++--
.../spark/procedure/CompactProcedureTestBase.scala | 136 +++++++++++++
51 files changed, 2081 insertions(+), 170 deletions(-)
mode change 100644 => 100755 .github/workflows/paimon-python-checks.yml
create mode 100644
paimon-core/src/test/java/org/apache/paimon/JavaPyE2ETest.java
create mode 100755 paimon-python/dev/run_mixed_tests.sh
mode change 100644 => 100755 paimon-python/pypaimon/api/rest_api.py
mode change 100644 => 100755 paimon-python/pypaimon/common/identifier.py
create mode 100644 paimon-python/pypaimon/common/memory_size.py
mode change 100644 => 100755 paimon-python/pypaimon/schema/data_types.py
mode change 100644 => 100755
paimon-python/pypaimon/snapshot/catalog_snapshot_commit.py
create mode 100755 paimon-python/pypaimon/tests/data_types_test.py
copy paimon-python/pypaimon/{api => tests/e2e}/__init__.py (100%)
create mode 100644 paimon-python/pypaimon/tests/e2e/java_py_read_write_test.py
copy paimon-python/pypaimon/{catalog => tests/manifest}/__init__.py (100%)
create mode 100644
paimon-python/pypaimon/tests/manifest/manifest_schema_test.py
mode change 100644 => 100755 paimon-python/pypaimon/tests/rest/rest_server.py
create mode 100644 paimon-python/pypaimon/tests/write/table_write_test.py
rename paimon-python/pypaimon/write/{batch_table_commit.py => table_commit.py}
(86%)
rename paimon-python/pypaimon/write/{batch_table_write.py => table_write.py}
(89%)
rename paimon-python/pypaimon/write/{batch_write_builder.py =>
write_builder.py} (73%)
