fabientra commented on issue #13570: URL: https://github.com/apache/gluten/issues/13570#issuecomment-6839506267
**Correction, and a narrower trigger: position deletes** My report says there are no delete files in the partition. That is wrong: I checked with Athena's `$files`, which does not list delete files. The table is merge-on-read (`write.delete.mode=merge-on-read`), and a delete commit added position delete files: - At the snapshot we read, the partition has 69 position delete files (20,958 positions). 14 of the 15 data files of the query have position deletes (8,551 positions). The file that held all the lost rows in our first analysis has 1,060 deleted positions. - The 7,083-row gap between the plain Parquet and the Iceberg counts is these deleted rows: the per-file counts before and after the delete commit differ by 7,083 in total. - So the plain Parquet comparison in the report did not exercise delete handling. New test: same engine (the 1.8.0-SNAPSHOT nightly, settings as in the report), same 15 data files, same query, alternating two snapshots with `VERSION AS OF`: | Snapshot | Gluten off | Gluten native, 5 runs | |---|---|---| | before the delete commit (no delete files) | 39,132,154 | 39,132,154 in every run, same `id_hash` | | after the delete commit (position deletes on 14 of the 15 files) | 39,125,071 | −84, −63, −9, −38, −334 | So rows are lost only when position deletes apply. The data files themselves are read correctly. The lost sets repeat across runs and versions: the −9 and −38 runs have the same `id_hash` as earlier runs, including a run on the 1.7.0-SNAPSHOT nightly. It looks like some rows are wrongly marked as deleted. The likely area is the position-delete path of the Iceberg split reader: `IcebergSplitReader::next` → `PositionalDeleteFileReader::readDeletePositions` / `updateDeleteBitmap` (Velox `IBM/velox`, branch `dft-2026_10_23`). Details: format v2 position delete files, each scoped to one data file (`referenced_data_file`). The file with the lost rows is 520 MB with 5 row groups. The query pushes filters on `app_id`, `platform` and `name`, and a timestamp range. The Velox Iceberg doc lists position deletes on partitioned tables as "Fallback mostly", but here the scan with position deletes runs natively. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
