Stefanietry opened a new issue, #6313: URL: https://github.com/apache/paimon/issues/6313
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Chain table is a new table type designed for data warehouse where full data are stored periodically, but the incremental change between periods is small. The primary benefits of using chain table include incremental computation and storage、significant storage savings、accelerated data pipelines etc ### Solution On the basis of the warehouse, two branches, delta and snapshot, are newly added to describe the newly changed data and the full data generated by chain compact respectively. During writing, data is written to the corresponding branch according to conf about branch, and during reading, the corresponding reading strategy is adopted based on whether full data exists in the current partition; Let's take the daily ODS Binlog Dump scene & weekly compact cycle as an example, and uses t$delta and t$snapshot to represent the delta and snapshot branches respectively, to introduce the layout, compact, writing and reading operation about the chain table. ### Anything else? [https://cwiki.apache.org/confluence/display/PAIMON/PIP-37%3A+Introduce+Chain+Table](url) ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
