dramaticlly opened a new pull request, #17706: URL: https://github.com/apache/iceberg/pull/17706
Follow up removal of #14045, this removes ability of writing Position Deletes With Row (PDWR) in java reference implementation. https://iceberg.apache.org/spec/#position-delete-files-with-row-data After the change, the Java implementation will no longer write PDWR delete files. It will only support reading the position and file name columns from previously created delete files or those generated by other engines. ### Core - `PositionDelete.set(CharSequence, long, R)` and getter of `row()` removed while keep the `size() == 3` and the positional `get`/`set` so that delete files already containing row data still read back correctly. - `Avro`, `ORC` and `Parquet` `buildPositionWriter()` now throw `UnsupportedOperationException` when a `rowSchema` is set. - `rowSchema` itself stays on all three `DeleteWriteBuilder`s as it is still **required** for equality deletes - `Avro.PositionAndRowDatumWriter` deleted; the `PositionDeleteStructWriter`s in `GenericOrcWriters` and `ParquetValueWriters` no longer expose the row column, and ORC's row-not-null precondition is dropped - `ORC`/`Parquet` `transformPaths` removed. Its only reader was the row-data branch - `RewriteTablePathUtil.PositionDeleteReaderWriter`: the `rowSchema` overload is gone and the 4-arg `writer()` is now abstract. ### Engines specific - `GenericFileWriterFactory`: both deprecated constructors and `Builder.positionDeleteRowSchema` removed, along with the `newPositionDeleteWriter` override that existed only to honour the row schema. The surviving constructor **keeps `writerProperties`** — `Builder.build()` sets it and it has live callers in Flink `RowDataTaskWriterFactory` and kafka-connect `RecordUtils`. Also removed 9 vestigial `configure*` overrides which override nothing now that `BaseFileWriterFactory` is gone (a `FormatModelRegistry` leftover rather than a position-delete concern, but in the same file). - `SparkFileWriterFactory` ×3, `SparkPositionDeltaWrite` ×3, `FlinkAppenderFactory` ×3: deprecated constructors and row-schema wiring removed; `FlinkAppenderFactory.newPosDeleteWriter` now builds path+pos writers. - `SparkPositionDeletesRewrite` ×3: `DeleteWriter` collapses from two writer factories to one. - `RewriteTablePathSparkAction` ×3: `SparkPositionDeleteReaderWriter` implements the 4-arg `writer()`. ### Behavior changes worth a close look 1. **Reads of `row` stay, writes are gone.** The `position_deletes` metadata table still has a `row` column for backward compatibility with existing v2 delete files; it is null for anything written from now on. 2. **Path rewriting drops the row column.** `RewriteTablePathUtil` previously detected row data in a source delete file and reproduced it; it can no longer do so, so rewritten position delete files contain path+pos only. Same for the `rewrite_position_delete_files` path in `SparkPositionDeletesRewrite`, which no longer propagates incoming rows. 3. **Existing callers that set a row schema now fail loudly** instead of writing rows. Follows the #13870's approach to fail early ### `position_deletes` metadata table and tests The table keeps exposing its `row` column so that delete files written before 1.12.0 still read back. Nothing can populate it any more, so it reads as null for anything written by this version. Related test which writes PDWR gets dropped ### Verification `gradlew check` `.palantir/revapi.yml` gains 16 entries across core, data, orc and parquet ### Extra Callout https://github.com/apache/iceberg/commit/d39351bed8395882ef8d2395f8c991d636be21af include change for `PositionDeltaWriter.delete(CharSequence, long, T, PartitionSpec, StructLike)` still takes a `row` parameter that is now ignored by `BasePositionDeltaWriter`. It is not deprecated, so narrowing it belongs in a 1.13.0 deprecation cycle. ## AI Disclosure Model: Claude Opus 5 (1M context) Platform/Tool: Claude Code Human Oversight: reviewed Prompt Summary: split #16449 into smaller self-contained PRs; verify each group compiles and tests green standalone @pvary if you can help take a look at the change? CC @szehon-ho for changes relate to position delete metadata table and PDWR in rewrite table path changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
