Parquet storage is a major feature for Kylin, I have heard a lot of user are waiting on the future Kylin 4.0. Kylin team is working on this feature, we hope to provided a technical preview as soon as possible. I have open a ticket to keep track on this, here is the link https://issues.apache.org/jira/browse/KYLIN-4659 . If you is interested in its progress, you can watch that issue.
-- Best wishes to you ! From ：Xiaoxiang Yu At 2020-07-24 19:23:11, "ShaoFeng Shi" <[email protected]> wrote: >Hello, Kylin users, > >Regarding the Kylin Parquet storage, we hope to update the progress here. >At present, we have completed the main development work[1], design >document[2], and the benchmark. With the new architecture, Kylin is going >to be more efficient and be more cloud-friendly: fully on Spark, less >dependency on Hadoop stack, which made the DevOps easier. > >Here we discuss the future plan, which includes the two aspects. > >1. The plan for Kylin 4.0 > >In Kylin 3.x, we have released some important functions/features, such as >real-time analysis, Flink building engine, global dictionary with Hive, >etc. In the next phase, we hope to concentrate on the Parquet storage >engine and to release it in Kylin v4.0 within this year. In this period, >3.x will be keeping maintained for bug fix and security vulnerability, but >won't introduce big change or major features. > >2. Backward compatibility for HBase storage. > >When we develop the Parquet storage engine, we find it is very difficult to >make the Parquet and HBase engines co-exist. The codebase becomes very >complicated and ugly, inevitably bring big challenges to the maintenance >and release. Besides, as HBase has different APIs (v1.1, v2.0, besides, the >CDHs' are different from the community's'), which makes the testing and >release effort be doubled or tripled in the past years. > >So, we plan to remove the HBase storage engine in Kylin 4.0; The Kylin >metadata will also migrate to MySQL. For existing users, if you want to use >the HBase engine, then keep in the Kylin 3.x; If you want to upgrade to the >Parquet storage, a migration tool can be provided later (another discuss >thread). > >Welcome to tell us your concerns and suggestions! Thank you for your >participation. > >## Reference >[1] https://github.com/apache/kylin/tree/kylin-on-parquet-v2 >[2] >https://cwiki.apache.org/confluence/display/KYLIN/KIP-1%3A+Parquet+storage > >Best regards, > >Shaofeng Shi 史少锋 >Apache Kylin PMC >Email: [email protected] > >Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html >Join Kylin user mail group: [email protected] >Join Kylin dev mail group: [email protected]
