[jira] [Commented] (CALCITE-3340) Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
[
https://issues.apache.org/jira/browse/CALCITE-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937076#comment-16937076
]
Rui Wang commented on CALCITE-3340:
---
I see.
OK, so I will try more ideas to see if I can keep both operators of TUMBLE. The
new idea of making TUMBLE created in parser worth to try.
If it turns out that we cannot keep both operators with the same name, at least
I have proven by PR that having a different name for new TUMBLE works (maybe
just TUMBLE_table, or TUMBLE_table_function).
> Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
> -
>
> Key: CALCITE-3340
> URL: https://issues.apache.org/jira/browse/CALCITE-3340
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The goal of this JIRA is to generate a logical plan for the following query:
> {code:java}
> SELECT *
> FROM TABLE(TUMBLE(
> TABLE ORDERS,
> INTERVAL '10' MINUTE))
> {code}
> This SQL query does not have DESCRIPTOR included, which is being tracked and
> discussed by CALCITE-3339.
> I expect we should generate a logical plan from this query that is similar to
> the following:
> {code:java}
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3], wstart=[$4],
> wend=[$5])
> LogicalTableFunctionScan(invocation=[TUMBLE($3, 6:INTERVAL MINUTE)],
> rowType=[RecordType(TIMESTAMP(0) ROWTIME, INTEGER ID, VARCHAR(10) PRODUCT,
> INTEGER UNITS, TIMESTAMP(0) wstart, TIMESTAMP(0) wend)])
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3])
> LogicalTableScan(table=[[ORINOCO, ORDERS]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3340) Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
[
https://issues.apache.org/jira/browse/CALCITE-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937060#comment-16937060
]
Julian Hyde commented on CALCITE-3340:
--
I really don't know.
> Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
> -
>
> Key: CALCITE-3340
> URL: https://issues.apache.org/jira/browse/CALCITE-3340
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The goal of this JIRA is to generate a logical plan for the following query:
> {code:java}
> SELECT *
> FROM TABLE(TUMBLE(
> TABLE ORDERS,
> INTERVAL '10' MINUTE))
> {code}
> This SQL query does not have DESCRIPTOR included, which is being tracked and
> discussed by CALCITE-3339.
> I expect we should generate a logical plan from this query that is similar to
> the following:
> {code:java}
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3], wstart=[$4],
> wend=[$5])
> LogicalTableFunctionScan(invocation=[TUMBLE($3, 6:INTERVAL MINUTE)],
> rowType=[RecordType(TIMESTAMP(0) ROWTIME, INTEGER ID, VARCHAR(10) PRODUCT,
> INTEGER UNITS, TIMESTAMP(0) wstart, TIMESTAMP(0) wend)])
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3])
> LogicalTableScan(table=[[ORINOCO, ORDERS]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3340) Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
[
https://issues.apache.org/jira/browse/CALCITE-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937044#comment-16937044
]
Rui Wang commented on CALCITE-3340:
---
I can see many of the functions are coded into parser (e.g. SUBSTRING, TRIM,
etc.), and they are not looked up by name.
However, based on my test and my read from code, both old TUMBLE(the group
function) and new TUMBLE(table function) don't follow that, thus these two
operators are looked up by name(and that's why same "TUMBLE" as operator name
leads to operator overloading and I always observed ). Either same SqlKind
(e.g. SqlKind.TUMBLE) or different SqlKind(e.g. assign OTHER_FUNCTION to new
TUMBLE operator) does not change the lookup name.
So do you suggest I code both TUMBLE into parser, like other built-in operators?
> Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
> -
>
> Key: CALCITE-3340
> URL: https://issues.apache.org/jira/browse/CALCITE-3340
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The goal of this JIRA is to generate a logical plan for the following query:
> {code:java}
> SELECT *
> FROM TABLE(TUMBLE(
> TABLE ORDERS,
> INTERVAL '10' MINUTE))
> {code}
> This SQL query does not have DESCRIPTOR included, which is being tracked and
> discussed by CALCITE-3339.
> I expect we should generate a logical plan from this query that is similar to
> the following:
> {code:java}
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3], wstart=[$4],
> wend=[$5])
> LogicalTableFunctionScan(invocation=[TUMBLE($3, 6:INTERVAL MINUTE)],
> rowType=[RecordType(TIMESTAMP(0) ROWTIME, INTEGER ID, VARCHAR(10) PRODUCT,
> INTEGER UNITS, TIMESTAMP(0) wstart, TIMESTAMP(0) wend)])
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3])
> LogicalTableScan(table=[[ORINOCO, ORDERS]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3340) Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
[
https://issues.apache.org/jira/browse/CALCITE-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936315#comment-16936315
]
Julian Hyde commented on CALCITE-3340:
--
Built-in operators -- especially those referenced by the parser -- are not
looked up by name. For instance, the parser creates a call to
SqlStdOperatorTable.SUBSTRING, not a call to SqlIdentifier("SUBSTRING") that is
later resolved by the validator.
Do they need to have the same SqlKind value? It's worth questioning that
assumption.
> Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
> -
>
> Key: CALCITE-3340
> URL: https://issues.apache.org/jira/browse/CALCITE-3340
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The goal of this JIRA is to generate a logical plan for the following query:
> {code:java}
> SELECT *
> FROM TABLE(TUMBLE(
> TABLE ORDERS,
> INTERVAL '10' MINUTE))
> {code}
> This SQL query does not have DESCRIPTOR included, which is being tracked and
> discussed by CALCITE-3339.
> I expect we should generate a logical plan from this query that is similar to
> the following:
> {code:java}
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3], wstart=[$4],
> wend=[$5])
> LogicalTableFunctionScan(invocation=[TUMBLE($3, 6:INTERVAL MINUTE)],
> rowType=[RecordType(TIMESTAMP(0) ROWTIME, INTEGER ID, VARCHAR(10) PRODUCT,
> INTEGER UNITS, TIMESTAMP(0) wstart, TIMESTAMP(0) wend)])
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3])
> LogicalTableScan(table=[[ORINOCO, ORDERS]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3340) Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
[
https://issues.apache.org/jira/browse/CALCITE-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933642#comment-16933642
]
Rui Wang commented on CALCITE-3340:
---
[~julianhyde]
Regarding to one point you mentioned before:
{code:java}
I think you need new TUMBLE and HOP functions. Is it possible to keep the old
ones and add new ones? The old ones probably don't need to be visible in the
operator table, because calls to them are created by the parser (explicitly)
rather than by the validator (looking up by name).
{code}
I am testing if both new and old functions can both exist in operator table. I
am not sure if I understand the intention of design correctly, but seems like
built-in function is expected to not be overloaded. Basically it leads to a "no
function match" if keep both old and new TUMBLE operators (I could give code
execution details if you are interest in).
Thus I started to explore what you mentioned: "The old ones probably don't need
to be visible in the operator table". I tried to mark old TUMBLE as private
static or remove it completely, but they have all failed tests.
Do you have suggestions to how to keep both old and new TUMBLE operators(same
operator name, same kind)?
> Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
> -
>
> Key: CALCITE-3340
> URL: https://issues.apache.org/jira/browse/CALCITE-3340
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The goal of this JIRA is to generate a logical plan for the following query:
> {code:java}
> SELECT *
> FROM TABLE(TUMBLE(
> TABLE ORDERS,
> INTERVAL '10' MINUTE))
> {code}
> This SQL query does not have DESCRIPTOR included, which is being tracked and
> discussed by CALCITE-3339.
> I expect we should generate a logical plan from this query that is similar to
> the following:
> {code:java}
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3], wstart=[$4],
> wend=[$5])
> LogicalTableFunctionScan(invocation=[TUMBLE($3, 6:INTERVAL MINUTE)],
> rowType=[RecordType(TIMESTAMP(0) ROWTIME, INTEGER ID, VARCHAR(10) PRODUCT,
> INTEGER UNITS, TIMESTAMP(0) wstart, TIMESTAMP(0) wend)])
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3])
> LogicalTableScan(table=[[ORINOCO, ORDERS]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3340) Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
[
https://issues.apache.org/jira/browse/CALCITE-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931705#comment-16931705
]
Rui Wang commented on CALCITE-3340:
---
{code:java}
Is it required that table-valued functions extend
SqlTableValueFunctionWindowingOperator? I don't think so. I'm not sure that a
"windowing operator" is a thing that Calcite cares about.
{code}
During prototype, I found at least it's required to overwrite *SqlOperator.
argumentMustBeScalar* to mark table parameter is not scalar(because the table
parameter is not scalar, which matters in validator). A child class to
overwrite that function will be required. For built-in table-valued functions,
if there is a table parameter, it will need to extend this child class. The
class name does not really matter though.
{code:java}
I am a bit surprised that TUMBLE appears in switch statements alongside
OTHER_FUNCTION. Do we currently require user-defined table functions to be of
type OTHER_FUNCTION?
{code}
I think so. My understanding is user-defined table functions will be
"UnresolvedFunction" and eventually falls into "OTHER_FUNCTION" (Calcite's
parser and validator are very complicated. I wouldn't be surprised if I am
wrong on this answer).
I will address other comments in the next version of PR which can pass tests.
Will let you know once I finish the work.
> Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
> -
>
> Key: CALCITE-3340
> URL: https://issues.apache.org/jira/browse/CALCITE-3340
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The goal of this JIRA is to generate a logical plan for the following query:
> {code:java}
> SELECT *
> FROM TABLE(TUMBLE(
> TABLE ORDERS,
> INTERVAL '10' MINUTE))
> {code}
> This SQL query does not have DESCRIPTOR included, which is being tracked and
> discussed by CALCITE-3339.
> I expect we should generate a logical plan from this query that is similar to
> the following:
> {code:java}
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3], wstart=[$4],
> wend=[$5])
> LogicalTableFunctionScan(invocation=[TUMBLE($3, 6:INTERVAL MINUTE)],
> rowType=[RecordType(TIMESTAMP(0) ROWTIME, INTEGER ID, VARCHAR(10) PRODUCT,
> INTEGER UNITS, TIMESTAMP(0) wstart, TIMESTAMP(0) wend)])
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3])
> LogicalTableScan(table=[[ORINOCO, ORDERS]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3340) Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
[
https://issues.apache.org/jira/browse/CALCITE-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930915#comment-16930915
]
Julian Hyde commented on CALCITE-3340:
--
* Is it required that table-valued functions extendÂ
SqlTableValueFunctionWindowingOperator? I don't think so. I'm not sure that a
"windowing operator" is a thing that Calcite cares about.
* PlannerTest is the place for tests of the planner. Not for tests that merely
use the planner. Maybe add stream.iq, similar to join.iq, and run it using
CoreQuidemTest.
* I am a bit surprised that TUMBLE appears in switch statements alongside
OTHER_FUNCTION. Do we currently require user-defined table functions to be of
type OTHER_FUNCTION?
* I know this is a prototype, but it's easier to review if other stuff is
clean (tests pass, checkstyle passes).
> Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
> -
>
> Key: CALCITE-3340
> URL: https://issues.apache.org/jira/browse/CALCITE-3340
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The goal of this JIRA is to generate a logical plan for the following query:
> {code:java}
> SELECT *
> FROM TABLE(TUMBLE(
> TABLE ORDERS,
> INTERVAL '10' MINUTE))
> {code}
> This SQL query does not have DESCRIPTOR included, which is being tracked and
> discussed by CALCITE-3339.
> I expect we should generate a logical plan from this query that is similar to
> the following:
> {code:java}
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3], wstart=[$4],
> wend=[$5])
> LogicalTableFunctionScan(invocation=[TUMBLE($3, 6:INTERVAL MINUTE)],
> rowType=[RecordType(TIMESTAMP(0) ROWTIME, INTEGER ID, VARCHAR(10) PRODUCT,
> INTEGER UNITS, TIMESTAMP(0) wstart, TIMESTAMP(0) wend)])
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3])
> LogicalTableScan(table=[[ORINOCO, ORDERS]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3340) Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
[
https://issues.apache.org/jira/browse/CALCITE-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930897#comment-16930897
]
Rui Wang commented on CALCITE-3340:
---
[~julianhyde]
I have updated this JIRA to clarify my goal and also cleaned up #1457 (use
TUMBLE, add javadoc, cleanup unnecessary changes, etc.) Could you please take
another look?
> Make TUMBLE be accepted as an operand for "FROM TABLE()" in validator
> -
>
> Key: CALCITE-3340
> URL: https://issues.apache.org/jira/browse/CALCITE-3340
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The goal of this JIRA is to generate a logical plan for the following query:
> {code:java}
> SELECT *
> FROM TABLE(TUMBLE(
> TABLE ORDERS,
> INTERVAL '10' MINUTE))
> {code}
> This SQL query does not have DESCRIPTOR included, which is being tracked and
> discussed by CALCITE-3339.
> I expect we should generate a logical plan from this query that is similar to
> the following:
> {code:java}
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3], wstart=[$4],
> wend=[$5])
> LogicalTableFunctionScan(invocation=[TUMBLE($3, 6:INTERVAL MINUTE)],
> rowType=[RecordType(TIMESTAMP(0) ROWTIME, INTEGER ID, VARCHAR(10) PRODUCT,
> INTEGER UNITS, TIMESTAMP(0) wstart, TIMESTAMP(0) wend)])
> LogicalProject(ROWTIME=[$0], ID=[$1], PRODUCT=[$2], UNITS=[$3])
> LogicalTableScan(table=[[ORINOCO, ORDERS]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
