Yes, thanks to Michael and Karshit for their great work. I am reviewing now, and doing some fix up (e.g. lint, repositories) so that we could get it into master as a "beta" component. I'll add updates in https://issues.apache.org/jira/browse/CALCITE-2040.
On Wed, Apr 7, 2021 at 9:37 PM Fan Liya <liya.fa...@gmail.com> wrote: > > Hi Michael, > > Thanks for sharing the great work. > I believe it is important work for both communities. > > Best, > Liya Fan > > > On Thu, Apr 8, 2021 at 3:30 AM Michael Mior <mm...@apache.org> wrote: > > > Hi all, > > > > I wanted to share some work one of my (now former) students, Karshit > > Shah, has done with integrating Apache Arrow into Calcite. Karshit has > > written an Arrow adapter that's able to perform filtering and > > projections natively on Arrow data using Gandiva so these expressions > > can be JITed using LLVM. The pull request[0] needs some cleanup, but > > the code is in relatively good shape. > > > > Right now, the adapter only reads from files, but I think there are a > > number of exciting extensions to this that are possible. For example, > > Arrow has a client-server framework Flight which could be connected > > with Calcite, perhaps via Avatica. (Andy Grove was doing some work on > > this last year[1] although I'm not sure of the progress.) > > > > The biggest blocker on this is actually not the Calcite code, but the > > availability of a suitably built Arrow dependency with Gandiva along > > with the appropriate CI configuration. I opened a JIRA on the Arrow > > project with some more details[2]. > > > > I'd love some thoughts on the approach and some help in pushing this > > over the finish line. > > > > [0] https://github.com/apache/calcite/pull/2133 > > [1] > > https://mail-archives.apache.org/mod_mbox/calcite-dev/202002.mbox/%3cCAJEf=X5xvXLQpJkX_VjJk=TnNRwT52v0=p28sczmid1tyce...@mail.gmail.com%3e > > [2] https://issues.apache.org/jira/browse/ARROW-11135 > > -- > > Michael Mior > > mm...@apache.org > >
