[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17799478#comment-17799478 ] Ulrich Kramer commented on CALCITE-4188: I also created a PR for this issue: https://github.com/apache/calcite/pull/3562 I had overlooked this ticket here. The main difference between the two PR is how correlation variable are handled: - 3562 uses an offset of 1 to put the correlation variables into the DataContext - 2116 uses an enum to distinguish between correlation variables and dynamic variables, which is the more robust way. > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[
https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498508#comment-17498508
]
Stamatis Zampetakis commented on CALCITE-4188:
--
Just following up on the question of [~gavinray] in the [mailing
list|https://lists.apache.org/thread/62tydrh7c1ogljv8dln3j55g9n44256z] the main
point holding me back from merging this PR in master is the new APIs in
{{SqlString}}. More specifically, I am a bit concerned about bringing concepts
from the Rex-land to the Sql-land. Unfortunately I didn't find time to see if
and how can this be avoided but leaving a comment here in case someone is able
to figure things out and it further.
> support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
>
>
> Key: CALCITE-4188
> URL: https://issues.apache.org/jira/browse/CALCITE-4188
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.25.0
>Reporter: JIasen Sheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
> function
> ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode,
> when implementing,
> [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675]
> will throw null Exception
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262491#comment-17262491 ] JIasen Sheng commented on CALCITE-4188: --- I used Google Translate, so sometimes there are problems with the expression. If there are any doubts, I will try to explain :-) > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262490#comment-17262490 ] Ruben Q L commented on CALCITE-4188: Ok, thanks for the clarifications [~angelzouxin] and [~zabetak]. > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262489#comment-17262489 ] JIasen Sheng commented on CALCITE-4188: --- [~rubenql] EnumerableCorrelate over JDBC is not supported at the moment JdbcToEnumerableConverter will eventually generate executable sql statements from the plan, which will eventually be executed by jdbc. https://github.com/apache/calcite/blob/404f968f6079360621896e6ee20ec12488a40bc2/core/src/main/java/org/apache/calcite/adapter/jdbc/JdbcToEnumerableConverter.java#L107 https://github.com/apache/calcite/blob/404f968f6079360621896e6ee20ec12488a40bc2/core/src/main/java/org/apache/calcite/adapter/jdbc/JdbcToEnumerableConverter.java#L174 EnumerableCorrelate generates a value during the execution process. When batchSize>1, the value will undergo related changes. This process is to fill in the corresponding variable before the sql is executed. SELECT * FROM SCOTT.emp WHERE ename =EnumerableCorrelateValue1 And empno support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258257#comment-17258257 ] Stamatis Zampetakis commented on CALCITE-4188: -- [~rubenql] I think that EnumerableCorrelate over JDBC is not supported at the moment. From my understanding so far this PR tries to solve external correlation (batch or simple) by transforming to dynamic params. > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[
https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258243#comment-17258243
]
Ruben Q L commented on CALCITE-4188:
Sorry I am arriving a bit late to this discussion.
I am not familiar with JdbcToEnumerableConverter, but I am familiar with
EnumerableBatchNestedLoopJoin.
I am a bit surprised seeing correlated variables implemented as "special"
dynamic parameters (I am not saying that this is wrong). IMO, we can see
EnumerableBatchNestedLoopJoin as a generalization of EnumerableCorrelate
(correlate is just kind of a batch nested loop with {{batchSize=1}}). How is
EnumerableCorrelate currently implemented in JdbcToEnumerableConverter? (I
assume it does not use dynamic parameters). Couldn't
EnumerableBatchNestedLoopJoin somehow reuse the EnumerableCorrelate logic in
JdbcToEnumerableConverter?
> support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
>
>
> Key: CALCITE-4188
> URL: https://issues.apache.org/jira/browse/CALCITE-4188
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.25.0
>Reporter: JIasen Sheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
> function
> ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode,
> when implementing,
> [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675]
> will throw null Exception
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257243#comment-17257243 ] JIasen Sheng commented on CALCITE-4188: --- I updated the commit. I tried to convert correlates to entirely dynamic param, but I don’t have a better idea to unify correlate and explicit dynamic parameters. > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17256823#comment-17256823 ] JIasen Sheng commented on CALCITE-4188: --- I tried to convert correlate to sqlDynamicParam already.Explicit dynamic parameters are generated in the sql parsing stage and assigned the corresponding index. The implicitly dynamic parameters do not participate in this stage. Therefore, a unified index value cannot be generated and can only be processed according to the corresponding correlate during unparse. If we reuse sqlDynamicParam, we also need to make corresponding distinctions internally. Considering the large differences in processing, I think it would be better to use a new class. SELECT * FROM SCOTT.emp WHERE empno SELECT * FROM SCOTT.emp WHERE ename =? And empno support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[
https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17256638#comment-17256638
]
Stamatis Zampetakis commented on CALCITE-4188:
--
You opted to model correlated variables as dynamic parameters in the SQL
statement and I think that's a good idea; but from that point onwards I don't
see why we should distinguish them. Possibly we could avoid introducing new
classes such as {{SqlFieldAccessCorrelate}} if we used entirely dynamic params.
I could try to see if it's possible to eliminate SqlFieldAccessCorrelate
without making the code much harder to understand but I was wondering if you
tried that already.
> support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
>
>
> Key: CALCITE-4188
> URL: https://issues.apache.org/jira/browse/CALCITE-4188
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.25.0
>Reporter: JIasen Sheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
> function
> ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode,
> when implementing,
> [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675]
> will throw null Exception
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17256379#comment-17256379 ] JIasen Sheng commented on CALCITE-4188: --- Hey [~zabetak] , When assigning values to dynamic parameters in sql, the explicit dynamic parameters are assigned by DataContext, and the values of implicit dynamic parameters are generated by Correlate. Therefore, I think it is necessary to choose different processing logic by distinguishing the types of dynamic parameters. > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255203#comment-17255203 ] Stamatis Zampetakis commented on CALCITE-4188: -- Hey [~angelzouxin], I had a quick look in the PR and I was wondering why it is necessary to distinguish between explicit dynamic parameters coming from the user and implicit dynamic parameters coming from the correlation. Can you elaborate a bit on this decision? > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17182031#comment-17182031 ] Rui Wang commented on CALCITE-4188: --- Thank you [~angelzouxin] I will give a code review soon on #2116 > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4188) support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement
[ https://issues.apache.org/jira/browse/CALCITE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17181730#comment-17181730 ] JIasen Sheng commented on CALCITE-4188: --- Wait for [https://github.com/apache/calcite/pull/2116] to have a decision . > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > > > Key: CALCITE-4188 > URL: https://issues.apache.org/jira/browse/CALCITE-4188 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.25.0 >Reporter: JIasen Sheng >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > support EnumerableBatchNestedLoopJoin in JdbcToEnumerableConverter implement > function > ENUMERABLE_BATCH_NESTED_LOOP_JOIN_RULE make correlate RexNode, > when implementing, > [https://github.com/apache/calcite/blob/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L675] > will throw null Exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
