This is an automated email from the ASF dual-hosted git repository.
XiaoHongbo-Hope pushed a change to branch release-2.1
in repository https://gitbox.apache.org/repos/asf/paimon.git
from a2748340ef [filesystem] Isolate Jindo OSS presigning dependencies
(#9856)
new 01b9ba4685 [docs] Reorganize ecosystem guides and add SVG diagrams
(#9728)
new b78b253af6 [docs] Reorganize CDC ingestion guides and diagrams (#9729)
new b3b5660c38 [docs] Restructure Program API guides and add SVG diagrams
(#9730)
new 9e0c6d4b88 [docs] Reorganize Iceberg compatibility guides and add SVG
diagrams (#9732)
new bf0a7a6216 [docs] Reorganize migration guides and add SVG diagrams
(#9733)
new bc7f8e2a41 [spark] Convert EqualNullSafe(NaN) to isNaN in V2 filters
(#9720)
new 1d53921f83 [cdc] Fix computed column args upper-cased in
case-insensitive mode (#9717)
new bc07ea8d45 [core][python] Preserve historical field options during
time travel (#9697)
new dc63739ced [python][ray] Preserve row kind in TableRead.to_ray()
(#9693)
new 800b360dc7 [vortex] Do not push unrepresentable timestamp predicate
literals (#9675)
new 2730cfa1ed [common] Fix invalid DecimalType when extracting a variant
decimal (#9672)
new 2cd2cfcc34 [format] Resolve the ORC compression kind independent of
the default locale (#9671)
new e27594f573 [common][flink] Fix missing cast rule for
DATE/TIME/TIMESTAMP to bounded CHAR/VARCHAR (#9670)
new caf5b4a65e [core] Fix rewriting file index with multiple map keys of
one column (#9669)
new 3fc0246d60 [core] Retry snapshot existence checks after I/O failures
(#9707)
new 01dd0daee6 [bug] Fix Self-suppression when close rethrows the same
exception in FileIO.overwriteFileUtf8 (#9674) (#9679)
new f7ab81a155 [docs] Reorganize Learn Paimon guides and add SVG diagrams
(#9736)
new da772c4247 [docs] Reorganize project guides and add SVG diagrams
(#9734)
new a9a3d0f7eb [codegen] Fix map equaliser for keys without value-based
Java equality (#9738)
new 7152705145 [common] Stop answering value predicates from a truncated
BSI index (#9654)
The 20 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/docs/cdc-ingestion/action-configuration.mdx | 159 +++++
docs/docs/cdc-ingestion/flink-cdc.mdx | 159 ++---
docs/docs/cdc-ingestion/index.mdx | 172 ++----
docs/docs/cdc-ingestion/kafka-cdc.mdx | 532 ++++------------
docs/docs/cdc-ingestion/kafka-debezium-bson.mdx | 302 +++++++++
.../cdc-ingestion/{mongo-cdc.md => mongo-cdc.mdx} | 288 ++++-----
.../cdc-ingestion/{mysql-cdc.md => mysql-cdc.mdx} | 167 ++---
.../{postgres-cdc.md => postgres-cdc.mdx} | 102 ++--
docs/docs/cdc-ingestion/pulsar-cdc.md | 378 ------------
docs/docs/cdc-ingestion/pulsar-cdc.mdx | 243 ++++++++
docs/docs/cdc-ingestion/schema-evolution.md | 95 +++
docs/docs/ecosystem/amoro.md | 48 +-
docs/docs/ecosystem/connecting-engines.md | 91 +++
docs/docs/ecosystem/doris.md | 255 ++++----
docs/docs/ecosystem/hive.mdx | 369 +++++------
docs/docs/ecosystem/index.md | 88 +--
docs/docs/ecosystem/starrocks.md | 200 +++---
docs/docs/ecosystem/trino.md | 328 ++++------
docs/docs/iceberg/append-table.mdx | 179 +++---
docs/docs/iceberg/catalogs.md | 91 +++
docs/docs/iceberg/configurations.mdx | 35 +-
docs/docs/iceberg/{index.md => data-types.md} | 76 +--
docs/docs/iceberg/ecosystem.mdx | 141 ++---
docs/docs/iceberg/hive-catalog.md | 183 +++---
docs/docs/iceberg/iceberg-tags.md | 84 ++-
docs/docs/iceberg/index.md | 166 ++---
docs/docs/iceberg/primary-key-table.mdx | 364 +++++------
docs/docs/iceberg/rest-catalog.mdx | 172 +++---
docs/docs/learn-paimon/ai-pipelines.mdx | 253 ++++++++
docs/docs/learn-paimon/index.md | 43 ++
docs/docs/learn-paimon/scenario-guide.mdx | 675 +++++----------------
docs/docs/learn-paimon/small-files.mdx | 161 +++++
docs/docs/learn-paimon/understand-files.mdx | 558 +++++------------
docs/docs/migration/clone-to-paimon.md | 174 ++++--
docs/docs/migration/index.md | 39 ++
docs/docs/migration/migration-from-hive.mdx | 228 +++++--
docs/docs/migration/upsert-to-partitioned.mdx | 222 +++----
docs/docs/program-api/catalog-api.md | 388 +++++-------
docs/docs/program-api/cpp-api.md | 102 ++--
docs/docs/program-api/file-cache.mdx | 86 ++-
docs/docs/program-api/flink-api.mdx | 175 +++---
docs/docs/program-api/index.md | 43 ++
docs/docs/program-api/java-api.mdx | 426 +++----------
docs/docs/program-api/java-reading.md | 205 +++++++
docs/docs/program-api/java-types.md | 96 +++
docs/docs/program-api/java-writing.md | 182 ++++++
docs/docs/program-api/rest-api.mdx | 37 +-
docs/docs/program-api/rust-api.md | 25 +-
docs/docs/project/committer.md | 24 +-
docs/docs/project/contributing.md | 283 ++++-----
docs/docs/project/creating-a-release.md | 166 +----
docs/docs/project/download.mdx | 85 +--
docs/docs/project/index.md | 35 ++
docs/docs/project/publishing-a-release.md | 156 +++++
docs/docs/project/releases.md | 159 +++++
docs/docs/project/security.md | 12 +-
docs/docs/project/verifying-a-release-candidate.md | 28 +-
docs/docs/pypaimon/ray-joins.md | 2 +-
docs/generated/kafka_sync_database.html | 11 +-
docs/generated/kafka_sync_table.html | 9 +-
docs/generated/mongodb_functions.html | 1 -
docs/generated/mongodb_operator.html | 1 -
docs/generated/mongodb_path_example.html | 11 +-
docs/generated/mongodb_sync_database.html | 7 +-
docs/generated/mongodb_sync_table.html | 7 +-
docs/generated/mysql_sync_database.html | 9 +-
docs/generated/mysql_sync_table.html | 7 +-
docs/generated/other_functions.html | 7 +-
docs/generated/postgres_sync_table.html | 7 +-
docs/generated/pulsar_sync_database.html | 9 +-
docs/generated/pulsar_sync_table.html | 9 +-
docs/generated/temporal_functions.html | 1 -
docs/sidebars.js | 122 +++-
docs/static/img/amoro-paimon.png | Bin 164164 -> 0 bytes
docs/static/img/amoro-paimon.svg | 64 ++
docs/static/img/cdc-ingestion-flink-pipeline.svg | 43 ++
docs/static/img/cdc-ingestion-flinksql.png | Bin 353489 -> 0 bytes
docs/static/img/cdc-ingestion-schema-evolution.png | Bin 312401 -> 0 bytes
docs/static/img/cdc-ingestion-schema-evolution.svg | 65 ++
docs/static/img/cdc-ingestion-sync-modes.svg | 64 ++
docs/static/img/ecosystem-catalog-access.svg | 49 ++
docs/static/img/ecosystem-overview.svg | 56 ++
docs/static/img/iceberg-metadata-layout.svg | 52 ++
docs/static/img/iceberg-primary-key-visibility.svg | 53 ++
docs/static/img/iceberg-publication.svg | 56 ++
docs/static/img/learn-paimon-ai-pipeline.svg | 69 +++
docs/static/img/learn-paimon-file-lifecycle.svg | 65 ++
docs/static/img/learn-paimon-small-files.svg | 61 ++
docs/static/img/learn-paimon-streaming-commit.svg | 66 ++
docs/static/img/learn-paimon-table-choice.svg | 55 ++
docs/static/img/migration-clone-overwrite.svg | 52 ++
docs/static/img/migration-strategies.svg | 60 ++
docs/static/img/migration-tag-to-partition.svg | 60 ++
docs/static/img/program-api-local-cache.svg | 54 ++
docs/static/img/program-api-overview.svg | 63 ++
docs/static/img/program-api-read-flow.svg | 52 ++
docs/static/img/program-api-write-flow.svg | 56 ++
docs/static/img/project/contribution-workflow.svg | 55 ++
docs/static/img/project/release-artifacts.svg | 83 +++
docs/static/img/project/release-lifecycle.svg | 55 ++
.../apache/paimon/codegen/ScalarOperatorGens.scala | 15 +-
.../paimon/codegen/EqualiserCodeGeneratorTest.java | 175 ++++++
.../paimon/casting/DateToStringCastRule.java | 19 +-
.../paimon/casting/TimeToStringCastRule.java | 19 +-
.../paimon/casting/TimestampToStringCastRule.java | 17 +-
.../org/apache/paimon/data/variant/VariantGet.java | 10 +-
.../bsi/BitSliceIndexBitmapFileIndex.java | 44 +-
.../src/main/java/org/apache/paimon/fs/FileIO.java | 21 +-
.../DateTimeToCharacterStringCastRuleTest.java | 97 +++
.../paimon/data/variant/GenericVariantTest.java | 32 +
.../data/variant/PaimonShreddingUtilsTest.java | 21 +
.../bsi/BitSliceIndexBitmapFileIndexTest.java | 64 ++
.../apache/paimon/index/FileIndexProcessor.java | 5 +-
.../paimon/table/AbstractFileStoreTable.java | 67 +-
.../org/apache/paimon/utils/SnapshotManager.java | 46 +-
.../paimon/index/FileIndexProcessorTest.java | 72 +++
.../paimon/table/DataEvolutionFileIndexTest.java | 76 +++
.../table/TimeTravelSchemaEvolutionTest.java | 260 ++++++++
.../apache/paimon/utils/SnapshotManagerTest.java | 26 +
.../paimon/flink/action/cdc/ComputedColumn.java | 33 +-
.../flink/action/cdc/ComputedColumnUtils.java | 44 +-
.../action/cdc/format/AbstractRecordParser.java | 3 +-
.../cdc/mongodb/strategy/MongoVersionStrategy.java | 3 +-
.../flink/action/cdc/mysql/MySqlRecordParser.java | 4 +-
.../action/cdc/postgres/PostgresRecordParser.java | 3 +-
.../flink/action/cdc/ComputedColumnUtilsTest.java | 74 +++
.../cdc/kafka/KafkaCanalSyncTableActionITCase.java | 20 +-
.../apache/paimon/flink/SchemaChangeITCase.java | 31 +
.../apache/paimon/format/orc/OrcWriterFactory.java | 31 +-
.../paimon/format/orc/OrcWriterFactoryTest.java | 79 +++
.../pypaimon/read/datasource/ray_datasource.py | 9 +-
.../pypaimon/read/datasource/split_provider.py | 11 +-
paimon-python/pypaimon/read/table_read.py | 3 +
paimon-python/pypaimon/table/file_store_table.py | 39 +-
.../pypaimon/tests/ray_integration_test.py | 54 ++
.../pypaimon/tests/split_provider_test.py | 6 +
.../table/time_travel_schema_evolution_test.py | 187 ++++++
.../paimon/spark/SparkV2FilterConverter.scala | 2 +
.../spark/sql/SparkV2FilterConverterTestBase.scala | 25 +
.../format/vortex/VortexPredicateConverter.java | 13 +
.../vortex/VortexPredicateConverterTest.java | 70 +++
141 files changed, 8798 insertions(+), 5193 deletions(-)
create mode 100644 docs/docs/cdc-ingestion/action-configuration.mdx
create mode 100644 docs/docs/cdc-ingestion/kafka-debezium-bson.mdx
rename docs/docs/cdc-ingestion/{mongo-cdc.md => mongo-cdc.mdx} (50%)
rename docs/docs/cdc-ingestion/{mysql-cdc.md => mysql-cdc.mdx} (75%)
rename docs/docs/cdc-ingestion/{postgres-cdc.md => postgres-cdc.mdx} (64%)
delete mode 100644 docs/docs/cdc-ingestion/pulsar-cdc.md
create mode 100644 docs/docs/cdc-ingestion/pulsar-cdc.mdx
create mode 100644 docs/docs/cdc-ingestion/schema-evolution.md
create mode 100644 docs/docs/ecosystem/connecting-engines.md
create mode 100644 docs/docs/iceberg/catalogs.md
copy docs/docs/iceberg/{index.md => data-types.md} (54%)
create mode 100644 docs/docs/learn-paimon/ai-pipelines.mdx
create mode 100644 docs/docs/learn-paimon/small-files.mdx
create mode 100644 docs/docs/program-api/java-reading.md
create mode 100644 docs/docs/program-api/java-types.md
create mode 100644 docs/docs/program-api/java-writing.md
create mode 100644 docs/docs/project/publishing-a-release.md
create mode 100644 docs/docs/project/releases.md
delete mode 100644 docs/static/img/amoro-paimon.png
create mode 100644 docs/static/img/amoro-paimon.svg
create mode 100644 docs/static/img/cdc-ingestion-flink-pipeline.svg
delete mode 100644 docs/static/img/cdc-ingestion-flinksql.png
delete mode 100644 docs/static/img/cdc-ingestion-schema-evolution.png
create mode 100644 docs/static/img/cdc-ingestion-schema-evolution.svg
create mode 100644 docs/static/img/cdc-ingestion-sync-modes.svg
create mode 100644 docs/static/img/ecosystem-catalog-access.svg
create mode 100644 docs/static/img/ecosystem-overview.svg
create mode 100644 docs/static/img/iceberg-metadata-layout.svg
create mode 100644 docs/static/img/iceberg-primary-key-visibility.svg
create mode 100644 docs/static/img/iceberg-publication.svg
create mode 100644 docs/static/img/learn-paimon-ai-pipeline.svg
create mode 100644 docs/static/img/learn-paimon-file-lifecycle.svg
create mode 100644 docs/static/img/learn-paimon-small-files.svg
create mode 100644 docs/static/img/learn-paimon-streaming-commit.svg
create mode 100644 docs/static/img/learn-paimon-table-choice.svg
create mode 100644 docs/static/img/migration-clone-overwrite.svg
create mode 100644 docs/static/img/migration-strategies.svg
create mode 100644 docs/static/img/migration-tag-to-partition.svg
create mode 100644 docs/static/img/program-api-local-cache.svg
create mode 100644 docs/static/img/program-api-overview.svg
create mode 100644 docs/static/img/program-api-read-flow.svg
create mode 100644 docs/static/img/program-api-write-flow.svg
create mode 100644 docs/static/img/project/contribution-workflow.svg
create mode 100644 docs/static/img/project/release-artifacts.svg
create mode 100644 docs/static/img/project/release-lifecycle.svg
create mode 100644
paimon-common/src/test/java/org/apache/paimon/casting/DateTimeToCharacterStringCastRuleTest.java
create mode 100644
paimon-core/src/test/java/org/apache/paimon/table/TimeTravelSchemaEvolutionTest.java
create mode 100644
paimon-python/pypaimon/tests/table/time_travel_schema_evolution_test.py
