Wes, This is very exciting. Thanks for writing up the detailed document.
I think it is time for me to start brushing up on modern C++. Andy. On Thu, Dec 6, 2018 at 8:37 PM Wes McKinney <wesmck...@gmail.com> wrote: > hi folks, > > I jotted down some high level ideas about directions I'd like to push > the various parts of the project on the C++ side along with the > language bindings in Python, R, Ruby, and others. Many people may know > that I am building a not-for-profit open source development team to > focus on Apache Arrow (https://ursalabs.org/), so this document is > partly for my colleagues to organize some lower-level technical > discussions and planning in the Arrow JIRA. I'm interested from > feedback from the whole Arrow community, and we obviously would love > to have as many people as possible involved who have an interest in > the C++ libraries and their bindings. > > The simplified summary is that I would like to work toward an > embeddable in-memory query engine in C++ that can be used in all the > bindings. This can be used in numerous contexts, from data frame > libraries to streaming data transformation. As a simple example, we > could compile filter expressions with Gandiva and apply these to a > stream of record batches being materialized from a directory of > Parquet files. > > There's a lot of pieces that still have to fall into place to do this > in a sustainable and non-hacky way. > > > https://docs.google.com/document/d/12dWBniKW2JQ-5djE3SPjyQXVquCAEmLXVlb1dnhLhQ0/edit#heading=h.62rx18p423rw > > Looking forward to the feedback of others! > > Thanks > Wes >
