Hi All, I would like to propose GraphAr [1] as a new apache incubator project, and you can find the proposal [2] for more details.
GraphAr is an open source and language independent data file format designed for efficient graph data storage and retrieval. It aims at supplying a standard format for large-scale graph data storage and processing that can be used by diverse existing graph systems, such as Neo4j, Nebula Graph, and Apache HugeGraph, to reduce the overhead when various systems work together. Specifically, it provides the following features: 1. Efficient format: GraphAr maintains the CSR (Compressed Sparse Row) and CSC (Compressed Sparse Column) semantics of graph data, partitions the data into chunks for parallel access, and leverages high-performance columnar storage formats (Parquet and ORC) for data file organizing. 2. Out-of-core queries: GraphAr is designed for out-of-core scenarios, enabling the storage and querying of large-scale graphs outside of memory, such as in data lakes. 3. Cross-language support: GraphAr provides libraries in C++, Java, Scala with Spark, and Python with PySpark for generating, accessing, and transforming files in GraphAr format GraphAr is currently adopted in the production environment at Alibaba and used in Fabarta and TuGraph. The community is still in its early age but with good shape and diversity, with 5 core developers and 14 contributors from different organizations. Given the extensive need in the graph community for a standardized file format, we believe the developer and user communities will continue to grow. The proposed initial committers are interested in joining ASF, and believe they could reinforce extensive collaboration and build a more vibrant community following the Apache Way. I will help this project as the champion and mentor, and many thanks to our three other mentors: * Calvin Kirs (k...@apache.org) * tison (ti...@apache.org) * Xiaoqiao He (hexiaoq...@apache.org) Look forward to your feedback. Thanks. Best Regards, Yu [1] https://github.com/alibaba/GraphAr [2] https://cwiki.apache.org/confluence/display/INCUBATOR/GraphArProposal --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
