wangzhigang1999 opened a new issue, #10460: URL: https://github.com/apache/paimon/issues/10460
### Search before asking - [x] I searched existing issues; related work is linked below. Related work: [#6735](https://github.com/apache/paimon/issues/6735) raised PyPaimon RowKind support in December 2025. Its implementation PR, [#6736](https://github.com/apache/paimon/pull/6736), proposed an optional `row_kinds` argument and a `__row_kind__` column for Arrow writes, but was closed on May 24, 2026 without being merged. This issue tracks a narrower bug reproducible on the commit below: the existing `write_row` API loses the RowKind carried by `GenericRow` / `InternalRow` when no `rowkind.field` is configured. The intended fix preserves those object events without adding a public Arrow API parameter. [#10433](https://github.com/apache/paimon/issues/10433) concerns delete-count metadata when `_VALUE_KIND=3` is already persisted. Here, the DELETE object itself is persisted as `_VALUE_KIND=0`. ### Paimon version Master at `23e3b140748c70d13fa0f2f8c831900bb5d17fdc`. ### Compute Engine PyPaimon, Python 3.10.16, PyArrow 19.0.1, macOS arm64. The reproducer uses the Python writer and reader. ### Minimal reproduce step Create a primary-key table without `rowkind.field`. Write and commit an INSERT object, then write and commit a DELETE object for the same key: ```python from tempfile import TemporaryDirectory import pyarrow as pa import pyarrow.parquet as pq from pypaimon import CatalogFactory, Schema from pypaimon.table.row.generic_row import GenericRow from pypaimon.table.row.row_kind import RowKind with TemporaryDirectory() as warehouse: catalog = CatalogFactory.create({'warehouse': warehouse}) catalog.create_database('default', True) catalog.create_table( 'default.events', Schema.from_pyarrow_schema( pa.schema([pa.field('id', pa.int32(), nullable=False), ('v', pa.int32())]), primary_keys=['id'], options={ 'bucket': '1', 'write.native.enabled': 'false', 'commit.native.enabled': 'false', 'read.native.enabled': 'false', 'scan.native-plan.enabled': 'false', }, ), False, ) table = catalog.get_table('default.events') for kind in (RowKind.INSERT, RowKind.DELETE): builder = table.new_batch_write_builder() writer, commit = builder.new_write(), builder.new_commit() try: writer.write_row(GenericRow([1, 10], table.fields, kind)) messages = writer.prepare_commit() kinds = [ row['_VALUE_KIND'] for message in messages for file in message.new_files for row in pq.ParquetFile(file.file_path).read().to_pylist() ] print(kind.name, kinds) commit.commit(messages) finally: writer.close() commit.close() reader = table.new_read_builder() splits = reader.new_scan().plan().splits() print('Remaining rows:', reader.new_read().to_arrow(splits).to_pylist()) ``` On the affected commit, the output is: ```text INSERT [0] DELETE [0] Remaining rows: [{'id': 1, 'v': 10}] ``` ### What doesn't meet your expectations? The DELETE object should produce `_VALUE_KIND=3`, and the latest snapshot should contain no rows: ```text INSERT [0] DELETE [3] Remaining rows: [] ``` The Row-to-Arrow conversion preserves the business fields but drops the object's RowKind. Without `rowkind.field`, the writer then assigns INSERT to the row. Callers can also lose UPDATE_BEFORE and UPDATE_AFTER events, and the object event does not reach the pre-routing `ignore-delete` / `ignore-update-before` checks. ### Anything else? The same object-event loss affects the native Row input conversion. A fix should preserve the configured `rowkind.field` precedence, retain object events when no field is configured, and keep ordinary Arrow input's default INSERT behavior. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
