Thanks Shaofeng and team for the efforts! Its really interesting to see that Flink and Trino projects can get the benefits of Gluten and boost their performance. Much appreciated, if you could share more on this integration, perf_numbers, pullrequest_reference, jira_reference etc. Will Flink and Trino already have a plugin mechanism in place to seamlessly integrate with Gluten ?
Thanks, Rakesh On Sat, Dec 23, 2023 at 6:13 PM ShaoFeng Shi <shaofeng...@apache.org> wrote: > Hi Enrico, > > This is a good question. Actually I asked the same to the team when they > find me for coaching. > > As the proposal mentioned, Gluten is a middle layer between the JVM-based > SQL engine and native libraries. It connects the two sides, and build > common services like unified plan transformation, seamless native > integration, clear JNI interfaces, etc. Spark is the first engine it > supports now, but the framework and mechanism can also work for other > engines like Trino, Flink SQL in the future (the proposal also mentioned > this). Underlying Gluten, it can integrate with different libararies, like > Velox, Clickhouse and Arrow (not ready yet). So, its mission and scope is > different, that's why we put it as a new project. > > Best regards, > > Shaofeng Shi 史少锋 > Apache Kylin PMC, > Apache Incubator PMC, > Email: shaofeng...@apache.org > > Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html > Join Kylin user mail group: user-subscr...@kylin.apache.org > Join Kylin dev mail group: dev-subscr...@kylin.apache.org > > > > > Enrico Olivelli <eolive...@gmail.com> 于2023年12月22日周五 22:43写道： > > > Shaofeng, > > > > Il giorno ven 22 dic 2023 alle ore 13:59 ShaoFeng Shi > > <shaofeng...@apache.org> ha scritto: > > > > > > Hi IPMC members, > > > > > > I would like to propose a new project to the ASF incubator - Gluten. > > > > > > Gluten[1] is a middle layer responsible for offloading Apache Spark SQL > > > queries to native engines. This project aims to address the CPU > > > computational bottleneck to offload SparkSQL operators to native > engines > > in > > > data loading scenarios based on Apache Spark. > > > > Have you considered making this project a subproject of Apache Spark ? > > > > Enrico > > > > > > > > Here is the proposal - > > > https://cwiki.apache.org/confluence/display/INCUBATOR/GlutenProposal > > > > > > I would be the Champion of the project. I will mentor and help the > > project > > > through the incubator with Yu Li [l...@apache.org], Kent Yao [ > > y...@apache.org] > > > and Wenli Zhang [ovi...@apache.org] . > > > > > > We are open to hearing the feedback from the incubator. > > > > > > Best, > > > Shaofeng Shi. > > > > > > [1] https://oap-project.github.io/gluten/ > > > > > > Best regards, > > > > > > Shaofeng Shi 史少锋 > > > Apache Kylin PMC, > > > Apache Incubator PMC, > > > Email: shaofeng...@apache.org > > > > > > Apache Kylin FAQ: > https://kylin.apache.org/docs/gettingstarted/faq.html > > > Join Kylin user mail group: user-subscr...@kylin.apache.org > > > Join Kylin dev mail group: dev-subscr...@kylin.apache.org > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > >
