[jira] [Created] (CALCITE-2756) ForEachStatement generates invalid Java Code

[Julian Feinauer (JIRA)]

Julian Feinauer created CALCITE-2756:


 Summary: ForEachStatement generates invalid Java Code
 Key: CALCITE-2756
 URL: https://issues.apache.org/jira/browse/CALCITE-2756
 Project: Calcite
  Issue Type: Bug
Reporter: Julian Feinauer
Assignee: Julian Feinauer


Code generated by the ForEachStatement in org.apache.calcite.linq4j.tree looks 
like:

{code:java}
 for (i : list) {
...
  }
{code}

I.e. the Paramter Type for the looping parameter is missing.
Thus, this cannot be used for codegen because Janino fails then.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (CALCITE-2756) ForEachStatement generates invalid Java Code

[Julian Feinauer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CALCITE-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Feinauer closed CALCITE-2756.

Resolution: Fixed

I just observed that this Code does not (yet) exist in master, thus I fix it in 
the local branch I'm working on. 

> ForEachStatement generates invalid Java Code
> 
>
> Key: CALCITE-2756
> URL: https://issues.apache.org/jira/browse/CALCITE-2756
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Feinauer
>Assignee: Julian Feinauer
>Priority: Major
>
> Code generated by the ForEachStatement in org.apache.calcite.linq4j.tree 
> looks like:
> {code:java}
>  for (i : list) {
> ...
>   }
> {code}
> I.e. the Paramter Type for the looping parameter is missing.
> Thus, this cannot be used for codegen because Janino fails then.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807566#comment-16807566
 ] 

Julian Feinauer commented on CALCITE-2966:
--

First, thanks [~danny0405]... are we sure that this is a bug or could it be due 
to me doing something wrong with the API?
Can I supply you with more informations about my code?

> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807828#comment-16807828
 ] 

Julian Feinauer commented on CALCITE-2966:
--

Hey [~danny0405]... the full code is in my fork for this commit: 
https://github.com/JulianFeinauer/calcite/tree/ae8704ab44632ae4bbe51cafef2251920955b183
The bug appears when the "testMatch" Method in the JdbcTest class is run.
This is the fastest and easiest way to reproduce it, is this okay for you?

> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808411#comment-16808411
 ] 

Julian Feinauer commented on CALCITE-2966:
--

Hi [~danny0405]... this is the last commit for the branch where multiple people 
have been working on for a long time now (over 1 year).
So its not the commit itself, that is interesting but the state after the 
commit with all the ~200 commits in the branch.
If it would help you, I can do a several branch where I squash all of them 
together, but in this branch I want to keep the commits as the work was done by 
several people and I think it would not be right to "mask" that by a single 
squash commit from myself.
So either you checkout the commit in this branch or I prepare another squashed 
branch for you, what you like better?

> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814400#comment-16814400
 ] 

Julian Feinauer commented on CALCITE-2966:
--

Hey [~danny0405] it seems that there was indeed a Bug in Calcite in the 
RexImpTable.implementNullSemantics, there was Lines 1029ff
{code:java}
case FALSE:
  // v0 != null && v1 != null && f(v0, v1)
  for (Ord operand : Ord.zip(call.getOperands())) {
if (translator.isNullable(operand.e)) {
  list.add(
  translator.translate(
  operand.e, NullAs.IS_NOT_NULL));
  translator = translator.setNullable(operand.e, false);
}
  }
  list.add(implementCall(translator, call, implementor, nullAs));
  return Expressions.foldAnd(list);
{code}
which should be 
{code:java}
case FALSE:
  // v0 != null && v1 != null && f(v0, v1)
  for (Ord operand : Ord.zip(call.getOperands())) {
if (translator.isNullable(operand.e)) {
  list.add(Expressions.notEqual(translator.translate(
  operand.e, NullAs.IS_NOT_NULL), Expressions.constant(null)));
  translator = translator.setNullable(operand.e, false);
}
  }
  list.add(implementCall(translator, call, implementor, nullAs));
  return Expressions.foldAnd(list);
{code}

I fixed this in my branch and would submit this together with the PR I'm 
preparing for the match recognize.
But it would be cool if you could also check if this is the correct solution.

> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Feinauer reassigned CALCITE-2966:


Assignee: Julian Feinauer  (was: Danny Chan)

> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Julian Feinauer
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16815490#comment-16815490
 ] 

Julian Feinauer commented on CALCITE-1935:
--

Hi all, I have a MWE now and am currentlry trying to prepare a PR with merging 
back master, checkstyle, ... .

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16816305#comment-16816305
 ] 

Julian Feinauer commented on CALCITE-2966:
--

Hi [~danny0405] all good!
I tell you as soon as the PR is in good shape (have some issues with merge back 
from master) then you can check there. But I guess the snippets I showed above 
completly remove this issue.

> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Julian Feinauer
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16816643#comment-16816643
 ] 

Julian Feinauer commented on CALCITE-2966:
--

Hey [~danny0405] a short update on this... I am nearly finished with the 
preparation of the PR and I see some strange bahvior with the snippet I showed 
above.
If I do the "fix" I intended to do above it makes the code generation work for 
the situation I have with MR but a lot of other tests fail, probably because 
they rely on a different behavior.
So for me its pretty unclear how this should exactly work and I'm  bit confused.

> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Julian Feinauer
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16816658#comment-16816658
 ] 

Julian Feinauer commented on CALCITE-1935:
--

Hi [~julianhyde] thanks for your offer. And I think I have to come back to it.
I'm trying to prepare a PR for master but the branch is pretty huge as it 
builds on a lot of preivous work from you and others.
So, for example there are some tests failing where I do not know how to fix 
them properly.
Overall I would be grateful to have you looking into the code and help me to 
bring the efforts to a PR-eable state.

Some issues I currently have are:
- 8 or 9 MR related tests in SqlToRelConverterTest fail (which were implemented 
by [~ransom]) but only due to another representation in the string, so this 
should be easy to fix if one knows how this Test Setup works.
- The largest Issue I currently have is related to CALCITE-2966. From the 
Codegeneration I do it seems like there is a Bug in RexImplTable. But when I 
"fix" it so that the JdbcTest.testMatch test becomes green a whole lot of other 
tests fail.
I try to investigate this behavior since two or three days but I do not 
understand completly why this is the case. I think this could be some kind of 
edge case (where the others rely on Boolean fields or primitives or something) 
and I do not, or do something different.
- It would be good to check if there are "leftovers" of non MR related code in 
these branches which was introduced by me or previous authors while "playing 
around"

The branch I'm currently working on is 
https://github.com/JulianFeinauer/calcite/tree/1935-mr-prepare-pr.

I think if we manage to get everything "green" the next step is to go to the PR 
phase and check all my solutions and if they are fine or things should be 
solved differently. But first things first, I guess.

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16816857#comment-16816857
 ] 

Julian Feinauer commented on CALCITE-2966:
--

Hi [~danny0405], thanks for your comment and the explanation.
But if I run the Test JdbcTest.testMatch the original code generates invalid 
java (and Janino fails then).
The test uses the Employee Table and it generates the Java:

{code:java}
return p0.commission && p2.commission && p4.commission > p6.commission
{code}

which should be in this case


{code:java}
p0.commission != null && p2.commission != null && p4.commission > p6.commission
{code}

Can you see why this does not happen?


> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Julian Feinauer
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16816857#comment-16816857
 ] 

Julian Feinauer edited comment on CALCITE-2966 at 4/13/19 7:24 AM:
---

Hi [~danny0405], thanks for your comment and the explanation.
But if I run the Test JdbcTest.testMatch the original code generates invalid 
java (and Janino fails then).
The test uses the Employee Table and it generates the Java:

{code:java}
return p0.commission && p2.commission && p4.commission > p6.commission
{code}

which should be in this case (as comission is a Long Field)


{code:java}
return p0.commission != null && p2.commission != null && p4.commission > 
p6.commission
{code}

Can you see why this does not happen?



was (Author: julian.feinauer):
Hi [~danny0405], thanks for your comment and the explanation.
But if I run the Test JdbcTest.testMatch the original code generates invalid 
java (and Janino fails then).
The test uses the Employee Table and it generates the Java:

{code:java}
return p0.commission && p2.commission && p4.commission > p6.commission
{code}

which should be in this case


{code:java}
p0.commission != null && p2.commission != null && p4.commission > p6.commission
{code}

Can you see why this does not happen?


> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Julian Feinauer
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818341#comment-16818341
 ] 

Julian Feinauer commented on CALCITE-2966:
--

Hi [~danny0405], thank you SOO much for your detailed analysis of the problem. 
Indeed I never worked with the NullAs before, so please excuse my mistake.
So how do I propagate the given "nullAs"?
If I understand you correctly, the issue is that the IS_NOT_NULL which is the 
input gets "lost" in the process and so the translator.translate(node) does the 
wrong thing?
So how should I forward the nullAs to the translator or the translation?
I would be very grateful for a hint... as I'm still a bit confused...

> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Julian Feinauer
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818676#comment-16818676
 ] 

Julian Feinauer commented on CALCITE-2966:
--

[~danny0405] please excuse this statement.. but I could kiss you : )
I just tested your proposal and it works flawlessly.
I'm really sorry to made you that much of work for such a minor change.. but I 
think my understanding of the Linq internals is still very limitted, so I 
couldn't see it.

Thank you so much, I will close this issue now, ok?

> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Julian Feinauer
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (CALCITE-2966) Problem with Code Generation

[Julian Feinauer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CALCITE-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Feinauer resolved CALCITE-2966.
--
Resolution: Not A Problem

In fact, there was no Bug in Calcite but I simply "used" it wrong. Many thanks 
to [~danny0405] for the excellent help!

> Problem with Code Generation
> 
>
> Key: CALCITE-2966
> URL: https://issues.apache.org/jira/browse/CALCITE-2966
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Julian Feinauer
>Priority: Major
> Fix For: 1.20.0
>
>
> From the mailing list:
> Hi all,
> I have some problems with the code generation from Linq4j which I'm unable to 
> resolve myself.
> Basically, I want to translate a condition from Rex to a Linq4j expression to 
> use it in generated code.
> In my example the Condition is from Match Recognize and in SQL is: 
> `up."commission" > prev(up."commission")`.
>  
> ```
> RexBuilder rexBuilder = new RexBuilder(implementor.getTypeFactory());
> RexProgramBuilder rexProgramBuilder = new 
> RexProgramBuilder(physType.getRowType(), rexBuilder);
>  
> rexProgramBuilder.addCondition(entry.getValue());
>  
> final Expression condition = 
> RexToLixTranslator.translateCondition(rexProgramBuilder.getProgram(),
>           (JavaTypeFactory) getCluster().getTypeFactory(),
>           builder2,
>           inputGetter1,
>           implementor.allCorrelateVariables,
>           implementor.getConformance());
>  
> builder2.add(Expressions.return_(null, condition));
> ```
>  
> Here, the condition seems okay, it is: ">(PREV(UP.$4, 0), PREV(UP.$4, 1))",  
> so it should be a comparison of two variables (I rewrite the PREV with a 
> custom Input Getter".
> But, the generated code (for Janino) is:
>  
> ```
> Object p1 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p0 = 
> (org.apache.calcite.test.JdbcTest.Employee) p1;
> Object p3 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p2 = 
> (org.apache.calcite.test.JdbcTest.Employee) p3;
> Object p5 = row_.get($L4J$C$0_1);
> org.apache.calcite.test.JdbcTest.Employee p4 = 
> (org.apache.calcite.test.JdbcTest.Employee) p5;
> Object p7 = row_.get($L4J$C$1_1);
> org.apache.calcite.test.JdbcTest.Employee p6 = 
> (org.apache.calcite.test.JdbcTest.Employee) p7;
> return p0.commission && p2.commission && p4.commission > p6.commission;
> ```
>  
> This confuses me a lot as I do not know where the check for p0.commission and 
> p2.commission comes from.
> It seems that Linq4j adds them as it expects these variables to be nullable, 
> but I have no idea on how to avoid this.
> These fields are Numeric so I always get a compilation exception.
>  
> Can someone help me with this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16816658#comment-16816658
 ] 

Julian Feinauer edited comment on CALCITE-1935 at 4/16/19 11:20 AM:


Hi [~julianhyde] thanks for your offer. And I think I have to come back to it.
I'm trying to prepare a PR for master but the branch is pretty huge as it 
builds on a lot of preivous work from you and others.
So, for example there are some tests failing where I do not know how to fix 
them properly.
Overall I would be grateful to have you looking into the code and help me to 
bring the efforts to a PR-eable state.

Some issues I currently have are:
- 8 or 9 MR related tests in SqlToRelConverterTest fail (which were implemented 
by [~ransom]) but only due to another representation in the string, so this 
should be easy to fix if one knows how this Test Setup works.
- It would be good to check if there are "leftovers" of non MR related code in 
these branches which was introduced by me or previous authors while "playing 
around"

The branch I'm currently working on is 
https://github.com/JulianFeinauer/calcite/tree/1935-mr-prepare-pr.

I think if we manage to get everything "green" the next step is to go to the PR 
phase and check all my solutions and if they are fine or things should be 
solved differently. But first things first, I guess.


was (Author: julian.feinauer):
Hi [~julianhyde] thanks for your offer. And I think I have to come back to it.
I'm trying to prepare a PR for master but the branch is pretty huge as it 
builds on a lot of preivous work from you and others.
So, for example there are some tests failing where I do not know how to fix 
them properly.
Overall I would be grateful to have you looking into the code and help me to 
bring the efforts to a PR-eable state.

Some issues I currently have are:
- 8 or 9 MR related tests in SqlToRelConverterTest fail (which were implemented 
by [~ransom]) but only due to another representation in the string, so this 
should be easy to fix if one knows how this Test Setup works.
- The largest Issue I currently have is related to CALCITE-2966. From the 
Codegeneration I do it seems like there is a Bug in RexImplTable. But when I 
"fix" it so that the JdbcTest.testMatch test becomes green a whole lot of other 
tests fail.
I try to investigate this behavior since two or three days but I do not 
understand completly why this is the case. I think this could be some kind of 
edge case (where the others rely on Boolean fields or primitives or something) 
and I do not, or do something different.
- It would be good to check if there are "leftovers" of non MR related code in 
these branches which was introduced by me or previous authors while "playing 
around"

The branch I'm currently working on is 
https://github.com/JulianFeinauer/calcite/tree/1935-mr-prepare-pr.

I think if we manage to get everything "green" the next step is to go to the PR 
phase and check all my solutions and if they are fine or things should be 
solved differently. But first things first, I guess.

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>

[jira] [Assigned] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Feinauer reassigned CALCITE-1935:


Assignee: Julian Hyde  (was: Julian Feinauer)

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896198#comment-16896198
 ] 

Julian Feinauer commented on CALCITE-1935:
--

Hey [~julianhyde], I submitted a PR which needs a bit of polishing but I think 
I need some help for the final steps. See my mail on the list 
(https://lists.apache.org/thread.html/475eec9bc95fc05ddabe3721aa0fffd545f77bf4fc5f7a9ea79c690e@%3Cdev.calcite.apache.org%3E).
 The PR can be found here https://github.com/apache/calcite/pull/1343.

I hope that I can manage to get these first steps done with your (or others) 
help
Thanks already!

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897031#comment-16897031
 ] 

Julian Feinauer commented on CALCITE-1935:
--

[~julianhyde] I will do and thank you for your help. I think it would be ideal 
if you give me some feedback on the questions in my mail then I can continue. 
Or if you have the time, also do some changes on your own :)

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897535#comment-16897535
 ] 

Julian Feinauer commented on CALCITE-1935:
--

Hey, I think this is a good way and definitely agree with that as I'm eager to 
see how you would choose to solve specific problems and how it's done in a 
compliant way with the project. I hope that the code is not too hard to follow 
and please come back to me whenever there are problems or something is unclear!

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902397#comment-16902397
 ] 

Julian Feinauer commented on CALCITE-1935:
--

Hey [~julianhyde], you assigned the issue back to me although I assigned it to 
you as its up to you to do the code review (which is what has to be done now). 
But please ping me, whenever I can assist you or answer questions.

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Feinauer
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902823#comment-16902823
 ] 

Julian Feinauer commented on CALCITE-1935:
--

Hey [~julianhyde], I just checked your branch and agree with all of your 
changes I've seen so far.
I also agree with your proposal to squash, I just did not do it for the exact 
same reason... that there where multiple authos involved and I wanted to keep 
the history for the final polishing. And I agree that most of the work was done 
by either you or Zianghe as it was in excellent shape already when I took over, 
so lets do it like you suggest.

PS.: Did you also check the points from my email, where an explicit look is 
needed?

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Feinauer
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903794#comment-16903794
 ] 

Julian Feinauer commented on CALCITE-1935:
--

Hey [~julianhyde] thank for your detailed response, just wanted to ensure that 
everything is tackled before we consider merging to master. I will look into 
your Quidem Test and try to figure it out and make it work!
I'll do this based off on your branch so that we can easily merge it back to 
your work.

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Feinauer
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903819#comment-16903819
 ] 

Julian Feinauer commented on CALCITE-1935:
--

I just created a PR where I added a Test to match.iq that works (and is 
structurally similar to yours). Perhaps this helps to get your working?
I cannot reproduce the compilation error : /

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Feinauer
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16904354#comment-16904354
 ] 

Julian Feinauer commented on CALCITE-1935:
--

The commit is
https://github.com/JulianFeinauer/calcite/commit/7c1fc2b4e6b01a625919d7ab9519e1218533010a

Regarding your comment with CircularArrayList vs my memory class I do not 
object. Better solutions should always be used. My work is not the end of MR I 
only tried to bring all loose ends together and bring it one step further for 
others to jump in and improve. 

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Feinauer
>Priority: Major
>  Labels: match
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905538#comment-16905538
 ] 

Julian Feinauer commented on CALCITE-1935:
--

Thank you so much [~julianhyde] for your support and the final polishing. I am 
very happy that I was finally able to contribute something 'major' to calcite! 
I hope that I'm able to improve MATCH_RECOGNIZE in the future. 

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Feinauer
>Priority: Major
>  Labels: match
> Fix For: 1.21.0
>
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (CALCITE-1935) Reference implementation for MATCH_RECOGNIZE

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905847#comment-16905847
 ] 

Julian Feinauer commented on CALCITE-1935:
--

Thank you very much [~danny0405]! 

> Reference implementation for MATCH_RECOGNIZE
> 
>
> Key: CALCITE-1935
> URL: https://issues.apache.org/jira/browse/CALCITE-1935
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Feinauer
>Priority: Major
>  Labels: match
> Fix For: 1.21.0
>
>
> We now have comprehensive support for parsing and validating MATCH_RECOGNIZE 
> queries (see CALCITE-1570 and sub-tasks) but we cannot execute them. I know 
> the purpose of this work is to do CEP within Flink, but a reference 
> implementation that works on non-streaming data would be valuable.
> I propose that we add a class EnumerableMatch that can generate Java code to 
> evaluate MATCH_RECOGNIZE queries on Enumerable data. It does not need to be 
> efficient. I don't mind if it (say) buffers all the data in memory and makes 
> O(n ^ 3) passes over it. People can make it more efficient over time.
> When we have a reference implementation, people can start playing with this 
> feature. And we can start building a corpus of data sets, queries, and their 
> expected result. The Flink implementation will be able to test against those 
> same queries, and should give the same results, even though Flink will be 
> reading streaming data.
> Let's create {{match.iq}} with the following query based on 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1:
> {code}
> !set outputformat mysql
> !use match
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  PARTITION BY product
>  ORDER BY tstamp
>  MEASURES  STRT.tstamp AS start_tstamp,
>LAST(UP.tstamp) AS peak_tstamp,
>LAST(DOWN.tstamp) AS end_tstamp,
>MATCH_NUMBER() AS mno
>  ONE ROW PER MATCH
>  AFTER MATCH SKIP TO LAST DOWN
>  PATTERN (STRT UP+ FLAT* DOWN+)
>  DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>FLAT AS FLAT.units_sold = PREV(FLAT.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> PRODUCTSTART_TSTAM PEAK_TSTAMP END_TSTAMP MNO
> -- --- --- --- --
> TWINKIES   01-OCT-2014 03-OCT-2014 06-OCT-2014  1
> TWINKIES   06-OCT-2014 08-OCT-2014 09-OCT-2014  2
> TWINKIES   09-OCT-2014 13-OCT-2014 16-OCT-2014  3
> TWINKIES   16-OCT-2014 18-OCT-2014 20-OCT-2014  4
> 4 rows selected.
> !ok
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Created] (CALCITE-3294) Implement FINAL Clause for MATCH_RECOGNIZE

[Julian Feinauer (Jira)]

Julian Feinauer created CALCITE-3294:


 Summary: Implement FINAL Clause for MATCH_RECOGNIZE
 Key: CALCITE-3294
 URL: https://issues.apache.org/jira/browse/CALCITE-3294
 Project: Calcite
  Issue Type: Improvement
Reporter: Julian Feinauer


With CALCITE-1935 the initial support for the MATCH_RECOGNIZE clause was 
introduced. But it is still lacking several features.
One of them is the FINAL Clause which forces the `MEASURE` to act global on all 
Tuples which were matched by the pattern.

See 
https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1

An example query would be:

{code}
SELECT *
FROM sales_history MATCH_RECOGNIZE (
 ORDER BY tstamp
 MEASURES  
   LAST(A.tstamp) AS ts_prev,
   FINAL LAST(A.tstamp) AS ts_last
 ALL ROWS PER MATCH
 PATTERN (A+)
 DEFINE
   A AS A.units_sold > 10
   ) MR
ORDER BY MR.product, MR.start_tstamp;
{code}

Here, the query matches for each sequence of rows which all have `units_sold > 
10`.
For the column `ts_prev` it always shows the timestamp of the timestamp of the 
previous match (the row before).
But for `ts_last` it shows the SAME value for each column as the `FINAL` 
modifier changes teh behavior to apply the `LAST` operator to the record set 
(similar to a window aggregation on the machted subset of rows).



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3294) Implement FINAL Clause for MATCH_RECOGNIZE

[Julian Feinauer (Jira)]

[ 
https://issues.apache.org/jira/browse/CALCITE-3294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16915742#comment-16915742
 ] 

Julian Feinauer commented on CALCITE-3294:
--

[~julianhyde] I could imagine working on that (although I have not that much 
time atm, just as a warning). But I would like to have your insights or ideas 
on how to implement that. My first idea is to treat it somewhat similar to how 
the Window Aggreation works as it should be really similar under the covers. 
Any ideas?

> Implement FINAL Clause for MATCH_RECOGNIZE
> --
>
> Key: CALCITE-3294
> URL: https://issues.apache.org/jira/browse/CALCITE-3294
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Julian Feinauer
>Priority: Major
>
> With CALCITE-1935 the initial support for the MATCH_RECOGNIZE clause was 
> introduced. But it is still lacking several features.
> One of them is the FINAL Clause which forces the `MEASURE` to act global on 
> all Tuples which were matched by the pattern.
> See 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1
> An example query would be:
> {code}
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  ORDER BY tstamp
>  MEASURES  
>LAST(A.tstamp) AS ts_prev,
>FINAL LAST(A.tstamp) AS ts_last
>  ALL ROWS PER MATCH
>  PATTERN (A+)
>  DEFINE
>A AS A.units_sold > 10
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> {code}
> Here, the query matches for each sequence of rows which all have `units_sold 
> > 10`.
> For the column `ts_prev` it always shows the timestamp of the timestamp of 
> the previous match (the row before).
> But for `ts_last` it shows the SAME value for each column as the `FINAL` 
> modifier changes teh behavior to apply the `LAST` operator to the record set 
> (similar to a window aggregation on the machted subset of rows).



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (CALCITE-3302) Add support for CLASSIFIER() command in MATCH_RECOGNIZE

[Julian Feinauer (Jira)]

Julian Feinauer created CALCITE-3302:


 Summary: Add support for CLASSIFIER() command in MATCH_RECOGNIZE
 Key: CALCITE-3302
 URL: https://issues.apache.org/jira/browse/CALCITE-3302
 Project: Calcite
  Issue Type: Improvement
Reporter: Julian Feinauer
Assignee: Julian Feinauer


The CLASSIFIER() command simply returns the defined pattern classifier that was 
matched for the respective column.

A very simple test case which could be added to match.iq is
{code:java}
select *
from "hr"."emps" match_recognize (
order by "empid" desc
measures "commission" as c,
"empid" as empid,
CLASSIFIER() as cl
pattern (s up)
define up as up."commission" < prev(up."commission"));

CEMPID  CL
 - -
1000   100   S
 500   200  UP

!ok
{code}




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3302) Add support for CLASSIFIER() command in MATCH_RECOGNIZE

[Julian Feinauer (Jira)]

 [ 
https://issues.apache.org/jira/browse/CALCITE-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Feinauer updated CALCITE-3302:
-
Description: 
The CLASSIFIER() command simply returns the defined pattern classifier that was 
matched for the respective column.

A very simple test case which could be added to match.iq is
{code:java}
select *
from "hr"."emps" match_recognize (
order by "empid" desc
measures "commission" as c,
"empid" as empid,
CLASSIFIER() as cl
pattern (s up)
define up as up."commission" < prev(up."commission"));

CEMPID CL
 - --
1000   100 S
 500   200 UP

!ok
{code}


  was:
The CLASSIFIER() command simply returns the defined pattern classifier that was 
matched for the respective column.

A very simple test case which could be added to match.iq is
{code:java}
select *
from "hr"."emps" match_recognize (
order by "empid" desc
measures "commission" as c,
"empid" as empid,
CLASSIFIER() as cl
pattern (s up)
define up as up."commission" < prev(up."commission"));

CEMPID  CL
 - -
1000   100   S
 500   200  UP

!ok
{code}



> Add support for CLASSIFIER() command in MATCH_RECOGNIZE
> ---
>
> Key: CALCITE-3302
> URL: https://issues.apache.org/jira/browse/CALCITE-3302
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Julian Feinauer
>Assignee: Julian Feinauer
>Priority: Major
>
> The CLASSIFIER() command simply returns the defined pattern classifier that 
> was matched for the respective column.
> A very simple test case which could be added to match.iq is
> {code:java}
> select *
> from "hr"."emps" match_recognize (
> order by "empid" desc
> measures "commission" as c,
> "empid" as empid,
> CLASSIFIER() as cl
> pattern (s up)
> define up as up."commission" < prev(up."commission"));
> CEMPID CL
>  - --
> 1000   100 S
>  500   200 UP
> !ok
> {code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3294) Implement FINAL Clause for MATCH_RECOGNIZE

[Julian Feinauer (Jira)]

[ 
https://issues.apache.org/jira/browse/CALCITE-3294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918044#comment-16918044
 ] 

Julian Feinauer commented on CALCITE-3294:
--

Found this pretty nice expansion 
https://docs.oracle.com/database/121/DWHSG/pattern.htm#GUID-F7ECC486-0BDA-47AD-90B9-C356F5E18C5B

> Implement FINAL Clause for MATCH_RECOGNIZE
> --
>
> Key: CALCITE-3294
> URL: https://issues.apache.org/jira/browse/CALCITE-3294
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Julian Feinauer
>Priority: Major
>
> With CALCITE-1935 the initial support for the MATCH_RECOGNIZE clause was 
> introduced. But it is still lacking several features.
> One of them is the FINAL Clause which forces the `MEASURE` to act global on 
> all Tuples which were matched by the pattern.
> See 
> https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1
> An example query would be:
> {code}
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
>  ORDER BY tstamp
>  MEASURES  
>LAST(A.tstamp) AS ts_prev,
>FINAL LAST(A.tstamp) AS ts_last
>  ALL ROWS PER MATCH
>  PATTERN (A+)
>  DEFINE
>A AS A.units_sold > 10
>) MR
> ORDER BY MR.product, MR.start_tstamp;
> {code}
> Here, the query matches for each sequence of rows which all have `units_sold 
> > 10`.
> For the column `ts_prev` it always shows the timestamp of the timestamp of 
> the previous match (the row before).
> But for `ts_last` it shows the SAME value for each column as the `FINAL` 
> modifier changes teh behavior to apply the `LAST` operator to the record set 
> (similar to a window aggregation on the machted subset of rows).



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3328) Immutable beans, powered by reflection

[Julian Feinauer (Jira)]

[ 
https://issues.apache.org/jira/browse/CALCITE-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924744#comment-16924744
 ] 

Julian Feinauer commented on CALCITE-3328:
--

I agree that this is a very good idea but also wonder why there is no default 
for something like that. Wouldnt this be something for apache commons as also 
other projects may need that?
And second question: Do we use custom annotations or do we use generic ones 
(JEE, ...) for that?

> Immutable beans, powered by reflection
> --
>
> Key: CALCITE-3328
> URL: https://issues.apache.org/jira/browse/CALCITE-3328
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> In quite a few places we have immutable config classes, and because they are 
> immutable we have builders. It all gets quite verbose.
> I propose to add a utility class, {{ImmutableBeans}}, that is able to 
> generate an implementation of a given interface. For example,
> {code:java}
> interface MyBean {
>   @Property  int getAge();
>   MyBean withAge(int age);
>   @Property String getName();
>   MyBean withName(String name);
> }
> MyBean b = ImmutableBeans.create(MyBean.class);
> assertThat(b.withAge(37).withName("Fred").getAge(), is(37));
> {code}
> We could replace {{SqlParser.ConfigBuilder}}, {{Frameworks.ConfigBuilder}}, 
> {{RelBuilder.ConfigBuilder}}, quite a few of the associated {{Config}} 
> classes, and perhaps quite a few fluent classes that we use in tests.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3328) Immutable beans, powered by reflection

[Julian Feinauer (Jira)]

[ 
https://issues.apache.org/jira/browse/CALCITE-3328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924748#comment-16924748
 ] 

Julian Feinauer commented on CALCITE-3328:
--

I see. Its probably debatable which approach fits better.
But generally I like to think about donating such things to commons (or else) 
as one may want to use it in other projects and its cleaner to include commons.

> Immutable beans, powered by reflection
> --
>
> Key: CALCITE-3328
> URL: https://issues.apache.org/jira/browse/CALCITE-3328
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
>
> In quite a few places we have immutable config classes, and because they are 
> immutable we have builders. It all gets quite verbose.
> I propose to add a utility class, {{ImmutableBeans}}, that is able to 
> generate an implementation of a given interface. For example,
> {code:java}
> interface MyBean {
>   @Property  int getAge();
>   MyBean withAge(int age);
>   @Property String getName();
>   MyBean withName(String name);
> }
> MyBean b = ImmutableBeans.create(MyBean.class);
> assertThat(b.withAge(37).withName("Fred").getAge(), is(37));
> {code}
> We could replace {{SqlParser.ConfigBuilder}}, {{Frameworks.ConfigBuilder}}, 
> {{RelBuilder.ConfigBuilder}}, quite a few of the associated {{Config}} 
> classes, and perhaps quite a few fluent classes that we use in tests.
> Work in progress: 
> https://github.com/julianhyde/calcite/tree/3328-immutable-beans



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3336) Intermittent failure of PigRelBuilderStyleTest.testImplWithGroupByCountDistinct

[Julian Feinauer (Jira)]

[ 
https://issues.apache.org/jira/browse/CALCITE-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16926140#comment-16926140
 ] 

Julian Feinauer commented on CALCITE-3336:
--

Thank you [~zabetak] for creating the issue, I added my configuration.

> Intermittent failure of 
> PigRelBuilderStyleTest.testImplWithGroupByCountDistinct
> ---
>
> Key: CALCITE-3336
> URL: https://issues.apache.org/jira/browse/CALCITE-3336
> Project: Calcite
>  Issue Type: Bug
>  Components: pig-adapter
>Affects Versions: 1.21.0
> Environment: +Jenkins environment+
> jdk-11-ea+28
> apache-maven-3.5.2
> Ubuntu 16.04 LTS
> +Local Environment (jfeinauer)+
> java version "1.8.0_181"
> Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 
> 2019-04-04T12:00:29-07:00)
> macOS Mojave 10.14.4 (18E226)
>Reporter: Stamatis Zampetakis
>Priority: Major
>
> The following tests fails intermittently with the following stack trace:
> {noformat}
> [ERROR] Tests run: 8, Failures: 0, Errors: 1, Skipped: 4, Time elapsed: 5.518 
> s <<< FAILURE! - in org.apache.calcite.test.PigRelBuilderStyleTest
> [ERROR] 
> testImplWithGroupByCountDistinct(org.apache.calcite.test.PigRelBuilderStyleTest)
>   Time elapsed: 3.82 s  <<< ERROR!
> org.apache.pig.impl.logicalLayer.FrontendException: Unable to open iterator 
> for alias t
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.assertScriptAndResults(PigRelBuilderStyleTest.java:270)
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.testImplWithGroupByCountDistinct(PigRelBuilderStyleTest.java:167)
> Caused by: org.apache.pig.PigException: Unable to store alias t
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.assertScriptAndResults(PigRelBuilderStyleTest.java:270)
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.testImplWithGroupByCountDistinct(PigRelBuilderStyleTest.java:167)
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: Error 
> processing rule LoadTypeCastInserter
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.assertScriptAndResults(PigRelBuilderStyleTest.java:270)
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.testImplWithGroupByCountDistinct(PigRelBuilderStyleTest.java:167)
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.assertScriptAndResults(PigRelBuilderStyleTest.java:270)
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.testImplWithGroupByCountDistinct(PigRelBuilderStyleTest.java:167)
> {noformat}
> The problem appeared a few times in Jenkins (e.g., in build 
> [1330|https://builds.apache.org/job/Calcite-Master/1330/jdk=JDK%2011%20(latest),label_exp=ubuntu&&!cloud-slave&&!H27&&!ubuntu-4/])
>  and was observed also by [~julian.feinauer].



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (CALCITE-3336) Intermittent failure of PigRelBuilderStyleTest.testImplWithGroupByCountDistinct

[Julian Feinauer (Jira)]

 [ 
https://issues.apache.org/jira/browse/CALCITE-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Feinauer updated CALCITE-3336:
-
Environment: 
+Jenkins environment+
jdk-11-ea+28
apache-maven-3.5.2
Ubuntu 16.04 LTS

+Local Environment (jfeinauer)+
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 
2019-04-04T12:00:29-07:00)
macOS Mojave 10.14.4 (18E226)

  was:
+Jenkins environment+
jdk-11-ea+28
apache-maven-3.5.2
Ubuntu 16.04 LTS


> Intermittent failure of 
> PigRelBuilderStyleTest.testImplWithGroupByCountDistinct
> ---
>
> Key: CALCITE-3336
> URL: https://issues.apache.org/jira/browse/CALCITE-3336
> Project: Calcite
>  Issue Type: Bug
>  Components: pig-adapter
>Affects Versions: 1.21.0
> Environment: +Jenkins environment+
> jdk-11-ea+28
> apache-maven-3.5.2
> Ubuntu 16.04 LTS
> +Local Environment (jfeinauer)+
> java version "1.8.0_181"
> Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 
> 2019-04-04T12:00:29-07:00)
> macOS Mojave 10.14.4 (18E226)
>Reporter: Stamatis Zampetakis
>Priority: Major
>
> The following tests fails intermittently with the following stack trace:
> {noformat}
> [ERROR] Tests run: 8, Failures: 0, Errors: 1, Skipped: 4, Time elapsed: 5.518 
> s <<< FAILURE! - in org.apache.calcite.test.PigRelBuilderStyleTest
> [ERROR] 
> testImplWithGroupByCountDistinct(org.apache.calcite.test.PigRelBuilderStyleTest)
>   Time elapsed: 3.82 s  <<< ERROR!
> org.apache.pig.impl.logicalLayer.FrontendException: Unable to open iterator 
> for alias t
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.assertScriptAndResults(PigRelBuilderStyleTest.java:270)
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.testImplWithGroupByCountDistinct(PigRelBuilderStyleTest.java:167)
> Caused by: org.apache.pig.PigException: Unable to store alias t
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.assertScriptAndResults(PigRelBuilderStyleTest.java:270)
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.testImplWithGroupByCountDistinct(PigRelBuilderStyleTest.java:167)
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: Error 
> processing rule LoadTypeCastInserter
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.assertScriptAndResults(PigRelBuilderStyleTest.java:270)
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.testImplWithGroupByCountDistinct(PigRelBuilderStyleTest.java:167)
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.assertScriptAndResults(PigRelBuilderStyleTest.java:270)
>   at 
> org.apache.calcite.test.PigRelBuilderStyleTest.testImplWithGroupByCountDistinct(PigRelBuilderStyleTest.java:167)
> {noformat}
> The problem appeared a few times in Jenkins (e.g., in build 
> [1330|https://builds.apache.org/job/Calcite-Master/1330/jdk=JDK%2011%20(latest),label_exp=ubuntu&&!cloud-slave&&!H27&&!ubuntu-4/])
>  and was observed also by [~julian.feinauer].



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (CALCITE-3341) Implement FINAL modifier functionality

[Julian Feinauer (Jira)]

Julian Feinauer created CALCITE-3341:


 Summary: Implement FINAL modifier functionality
 Key: CALCITE-3341
 URL: https://issues.apache.org/jira/browse/CALCITE-3341
 Project: Calcite
  Issue Type: Bug
Reporter: Julian Feinauer


And set default modifier to RUNNING that its compliant with Oracle 
implementation, see CALCITE-3302.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (CALCITE-3345) Implement time_bucket function

[Julian Feinauer (Jira)]

Julian Feinauer created CALCITE-3345:


 Summary: Implement time_bucket function
 Key: CALCITE-3345
 URL: https://issues.apache.org/jira/browse/CALCITE-3345
 Project: Calcite
  Issue Type: New Feature
Reporter: Julian Feinauer


See here for information on the `time_bucket` function: 
https://docs.timescale.com/latest/api#time_bucket

This is a more powerful version of the standard PostgreSQL date_trunc function. 
It allows for arbitrary time intervals instead of the second, minute, hour, 
etc. provided by date_trunc. The return value is the bucket's start time.

This would especially help with time averaging but keeps everything SQL 
compliant. E.g. queries like 

Example query from (https://www.timescale.com/):

{code:sql}
SELECT  time_bucket('10  seconds',  time)  AS  ten_second,
machine_id,  avg(temperature)  AS  "avgT",
min(temperature)  AS  "minT",  max(temperature)  AS  "maxT",
last(temperature,  time)  AS  "lastT"
FROM  measurements
WHERE  machine_id  =  'C931baF7'
AND  time  >  now()  -  interval  '150s'
GROUP  BY  ten_second
ORDER  BY  ten_second  DESC;
{code}




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CALCITE-3345) Implement time_bucket function

[Julian Feinauer (Jira)]

[ 
https://issues.apache.org/jira/browse/CALCITE-3345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929866#comment-16929866
 ] 

Julian Feinauer commented on CALCITE-3345:
--

Hi, didnt know that, thanks for the link, this looks good. But I would also 
like to implement this just for conformance with timescaleDB?

> Implement time_bucket function
> --
>
> Key: CALCITE-3345
> URL: https://issues.apache.org/jira/browse/CALCITE-3345
> Project: Calcite
>  Issue Type: New Feature
>Reporter: Julian Feinauer
>Priority: Major
>
> See here for information on the `time_bucket` function: 
> https://docs.timescale.com/latest/api#time_bucket
> This is a more powerful version of the standard PostgreSQL date_trunc 
> function. It allows for arbitrary time intervals instead of the second, 
> minute, hour, etc. provided by date_trunc. The return value is the bucket's 
> start time.
> This would especially help with time averaging but keeps everything SQL 
> compliant. E.g. queries like 
> Example query from (https://www.timescale.com/):
> {code:sql}
> SELECT  time_bucket('10  seconds',  time)  AS  ten_second,
> machine_id,  avg(temperature)  AS  "avgT",
> min(temperature)  AS  "minT",  max(temperature)  AS  "maxT",
> last(temperature,  time)  AS  "lastT"
> FROM  measurements
> WHERE  machine_id  =  'C931baF7'
> AND  time  >  now()  -  interval  '150s'
> GROUP  BY  ten_second
> ORDER  BY  ten_second  DESC;
> {code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (CALCITE-2640) Support for SQL on Timeseries / Traces

[Julian Feinauer (JIRA)]

Julian Feinauer created CALCITE-2640:


 Summary: Support for SQL on Timeseries / Traces
 Key: CALCITE-2640
 URL: https://issues.apache.org/jira/browse/CALCITE-2640
 Project: Calcite
  Issue Type: New Feature
Reporter: Julian Feinauer
Assignee: Julian Hyde


When working with time series data, e.g., from IoT devices, traces or other 
sources there is often the need for analysis on "transients". This means that 
it is important to compare values with their neighbors (i.e., prev / next) to 
detect jumps or changes of bits.

This is possible with the MATCH_RECOGNIZE functionality from SQL 2016 but 
perhaps there is also the possibility for a "sensible" extension to SQL (like 
Streaming SQL is) to support these use cases with a "less technical" syntax. 
Others also use a pseudo-SQL syntax to do "more time series focussed" things 
(like use GROUP BY to do time averaging in IinfluxDB). See [1] for InfluxDBs 
SQL-like QL InfluxQL.
And it would be better to have a "more standardized" way of the interaction 
between SQL and timeseries than several SQL-like DSLs.

There was also a discussion on the list, see [2].

[1] https://docs.influxdata.com/influxdb/v1.6/query_language/
[2] 
https://lists.apache.org/thread.html/250575a56165851ab55351b90a26eaa30e84d5bbe2b31203daaaefb9@%3Cdev.calcite.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CALCITE-1414) Add a RANDOM function

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15751039#comment-15751039
 ] 

Julian Feinauer commented on CALCITE-1414:
--

I added a pull request: https://github.com/apache/calcite/pull/338 with my 
suggestion how to implement this function.

> Add a RANDOM function
> -
>
> Key: CALCITE-1414
> URL: https://issues.apache.org/jira/browse/CALCITE-1414
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>  Labels: newbie
>
> Add a RANDOM function.
> It is not standard SQL, but most DBMSs have one, and it be useful.
> It will also allow us to better test non-deterministic expressions.
> I propose that:
> *  {{RANDOM(n)}} returns an integer between 0 and n - 1 inclusive.
> * There is no facility to specify a seed at this time.
> * The random number generator is randomly seeded at query start time based on 
> (say) the method used by Java's {{new Random()}}; if you run the query twice, 
> you will (probably) not get the same results.
> * If you call the function twice in the same row, it will (probably) return 
> different results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CALCITE-1414) Add a RANDOM function

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15751039#comment-15751039
 ] 

Julian Feinauer edited comment on CALCITE-1414 at 12/15/16 11:03 AM:
-

I added a pull request: https://github.com/apache/calcite/pull/339 with my 
suggestion how to implement this function.

I had to resubmit the PR https://github.com/apache/calcite/pull/338 due to a 
problem with the CI / Validator.
Sorry for that.




was (Author: julian.feinauer):
I added a pull request: https://github.com/apache/calcite/pull/338 with my 
suggestion how to implement this function.

> Add a RANDOM function
> -
>
> Key: CALCITE-1414
> URL: https://issues.apache.org/jira/browse/CALCITE-1414
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>  Labels: newbie
>
> Add a RANDOM function.
> It is not standard SQL, but most DBMSs have one, and it be useful.
> It will also allow us to better test non-deterministic expressions.
> I propose that:
> *  {{RANDOM(n)}} returns an integer between 0 and n - 1 inclusive.
> * There is no facility to specify a seed at this time.
> * The random number generator is randomly seeded at query start time based on 
> (say) the method used by Java's {{new Random()}}; if you run the query twice, 
> you will (probably) not get the same results.
> * If you call the function twice in the same row, it will (probably) return 
> different results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CALCITE-1414) Add a RANDOM function

[Julian Feinauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CALCITE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15760892#comment-15760892
 ] 

Julian Feinauer commented on CALCITE-1414:
--

[~julianhyde], I renamed everything to RAND_INTEGER and added a Comment in the 
Readme.md.
Please feel free to give me any more hints if I should change something.

> Add a RANDOM function
> -
>
> Key: CALCITE-1414
> URL: https://issues.apache.org/jira/browse/CALCITE-1414
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Feinauer
>  Labels: newbie
> Fix For: 1.11.0
>
>
> Add a RANDOM function.
> It is not standard SQL, but most DBMSs have one, and it be useful.
> It will also allow us to better test non-deterministic expressions.
> I propose that:
> *  {{RANDOM\(n)}} returns an integer between 0 and n - 1 inclusive.
> * There is no facility to specify a seed at this time.
> * The random number generator is randomly seeded at query start time based on 
> (say) the method used by Java's {{new Random()}}; if you run the query twice, 
> you will (probably) not get the same results.
> * If you call the function twice in the same row, it will (probably) return 
> different results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)