Dear Paimon Development Team,
We are a team of C++ developers who have recently developed a C++ version of
Paimon. I am reaching out to propose contributing this C++ version to the
Paimon community. Considering its unique architecture and functionalities, we
suggest that it may be advantageous to host it as a new sub-repo of Paimon,
potentially under the name paimon-cpp. Paimon-cpp aims to provide a native,
high-performance and extensible implementation that allows native engines to
access the Paimon datalake format. Some key features of the C++ version include:
1. Basic operations—including write, commit, scan, and read—implemented for
both append-only and primary key tables.
2. Interfaces for batch reading and writing utilizing the Arrow Columnar
In-Memory Format to enhance throughput.
3. A plugin mechanism enabling users to flexibly define file formats, file
systems, memory pools, thread pools, etc.
4. Default built-in plugins for ORC and Parquet file formats; local and OSS
file systems; integrated memory pool and thread pool plugins.
Furthermore, we assure compatibility with Paimon communication protocols and
meta formats, providing users with the flexibility to seamlessly switch between
multiple languages. We hope that the C++ version will contribute to the
enrichment of the Paimon community, enhancing its diversity and expanding its
capabilities.
I am excited about the possibility of collaborating with the community to
refine and expand this version further. Please let me know if the development
team is interested in this contribution or any specific procedures I need to
follow for submission.
Thank you for considering this proposal. I am looking forward to your rely.
Best regards,
Xinyu Liu
