yunfengzhou-hub opened a new issue, #10694: URL: https://github.com/apache/paimon/issues/10694
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Part of #10690. PIP-39 sends regular writes to the Lakestream Store but `INSERT OVERWRITE` to Paimon, so that backfilling the lake side does not go through the store. On a Flink lake-stream table such a statement is rejected instead, because the write is served by the store and the store cannot execute an overwrite. This issue is about `INSERT OVERWRITE` landing on Paimon as the PIP requires. ### Solution Required behaviour: - `INSERT OVERWRITE` on a lake-stream table replaces data in the Paimon table and not in the store, whether the partition is specified statically or left dynamic. - Regular writes on the same table keep reaching the store. Which side executes is decided per statement, not per table and not per session. - A lake-stream table keeps every write capability a Paimon table has: partitioned writes, row-level update and delete, and delete push-down. Where a statement needs something the store cannot do, it fails with an error that names the store, rather than being carried out on the wrong side. - `TRUNCATE TABLE` on a lake-stream table is rejected. The PIP does not mention it, and clearing the Paimon table alone would leave the store's copy of the same data behind, so there is no safe single-sided execution. - The behaviour is verified against a real Lakestream Store cluster, not only against a test double. ### Anything else? The PIP does not check whether the data being overwritten has already expired from the store, and leaves the resulting consistency to the user; that caveat stands and is not addressed here. ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
