Re: Issues in exposing data via TableFunction vs TableMacro

2019-09-11 Thread Gabriel Reid
Hi Julian, On Tue, Sep 10, 2019 at 5:19 PM Julian Feinauer < j.feina...@pragmaticminds.de> wrote: > > when going through the Code I just had another Idea. > Currently a TableFunction is executed as EnumerableTableFunctionScan which > gets generated from a LogicalTableFunctionScan by the Rule > En

Re: Issues in exposing data via TableFunction vs TableMacro

2019-09-10 Thread Julian Hyde
I always thought of a table function as a lightweight relational operator. You could write your own UNION function, for instance. But the moment you want it to start participating in algebraic rewrites - if you want to push filters through it, for instance - then you had better make it into a re

Re: Issues in exposing data via TableFunction vs TableMacro

2019-09-10 Thread Julian Feinauer
Hey, when going through the Code I just had another Idea. Currently a TableFunction is executed as EnumerableTableFunctionScan which gets generated from a LogicalTableFunctionScan by the Rule EnumerableTableFunctionScanRule. What if you just remove that Rule and add a custom Rule of yours which

Re: Issues in exposing data via TableFunction vs TableMacro

2019-09-10 Thread Julian Feinauer
Hi Gabriel, thats an interesting question for me too. Do you need parameters for those "dynamic tables"? If not you could do it similar to what Drill is doing and just implement a Schema which always returns "true" if someone asks for a Table and then returns a Table Implementation that you prov

Issues in exposing data via TableFunction vs TableMacro

2019-09-10 Thread Gabriel Reid
Hi, I'm currently using a combination of TableFunctions and TableMacros to expose various dynamic (relatively unstructured) data sources via Calcite. The underlying data sources are such that data can only be retrieved by first specifying what you want (i.e. there is no catalog of all data that is