I am working on a project which is a very simple version of a cube engine. I dont want to execute the SQL - we have our own execution frameworks. A clear separation between planner and executor will be helpful in the public APIs. In the first round of reading calcite internals the separation is not clear to me. I am not yet familiar with the code and may very well be making bad decisions.
A high level overview of my project (to give you a data point and quick design review) is: The project has to: 1. Detect certain patterns in a SQL query - tables it accesses, lookups, selective filters etc. 2. Rewrite the query if required. I think I've finally got 1) to work. My entry point is Schemas.convert(). I get a RelNode which I can visit and detect patterns. I havent yet tackled 2. Hopefully will get to it in a week or two. On Thu, Feb 12, 2015 at 12:43 PM, Vladimir Sitnikov < [email protected]> wrote: > Milinda> Samza's streaming query planning phase > > Can you clarify a bit? > If I understand right, you aim to "wire Samza's operators in an > optimized way and execute it as usual Samza's chain", don't you? > In that case, it looks like you need planner only. > > In case you want to stream Samza's results through java.sql.ResultSet, > then Enumerable/Bindable question makes sense. > > Milinda> Does this convention effect the query planning decisions other > than > transformations like LogicalTableScan to BindableTableScan > > It does not. > > Juilan> if people want to execute queries using just core > > Anybody out there wanting to execute queries using just core? > Can we have some requirements before we implement one more executor? > > In 584 the discussion is regarding "being able to use Calcite core as > planner only". > > Juilan> It will also make core smaller and simpler. > > It won't be simpler. Support of interpreter would take time. > Let's better have separation of modules like "planner" and "executor". > At least it would be a proof of ability to plug executor without > changing "hard-coded rules in the planner". > > Juilan> Most people find it easier to implement a bindable convention > > Please, do not mix API vs convention. > It does not require a new convention to process Scannable. > > Vladimir > -- Rajat Venkatesh | Engg Lead Qubole Inc | www.qubole.com
