[jira] [Created] (CALCITE-3562) Unify function's operands type check logic in validation and behavior in runtime
Feng Zhu created CALCITE-3562:
-
Summary: Unify function's operands type check logic in validation
and behavior in runtime
Key: CALCITE-3562
URL: https://issues.apache.org/jira/browse/CALCITE-3562
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Feng Zhu
Current now, there are many issues caused by inconsistency between validator
and runtime phase. To summarize:
(1)Validation phase allows a wide range of operand types, but the runtime
implementation does not cover all cases.
For example, _*SqlFunction(MOD)*_ adopts
_*OperandTypes.EXACT_NUMERIC_EXACT_NUMERIC*_.
{code:java}
@Test public void test0() {
final String sql = "SELECT mod(12.5, cast(1 as bigint))";
CalciteAssert.that()
.query(sql)
.returns("EXPR$0=0.5\n");
}{code}
We will get:
{code:java}
java.lang.RuntimeException: while resolving method 'mod[class
java.math.BigDecimal, long]' in class class
org.apache.calcite.runtime.SqlFunctions
at org.apache.calcite.linq4j.tree.Types.lookupMethod(Types.java:323)
at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:445)
at
org.apache.calcite.adapter.enumerable.RexImpTable$MethodNameImplementor.implement(RexImpTable.java:2253)
at
org.apache.calcite.adapter.enumerable.RexImpTable.implementCall(RexImpTable.java:1195){code}
(2)Type is assignable conceptually, but in the runtime phase, explicite cast is
still required.
For example, according to _*SqlTypeAssignmentRules*_, *_ST_MakePoint(Decimal,
Decimal)_* also accepts operands with (_Integer_, _Decimal_) types.
{code:java}
@Test public void test1() {
final String sql = "SELECT ST_MakePoint(1, 2.1)";
CalciteAssert.that()
.with(CalciteAssert.Config.GEO)
.query(sql)
.returns("EXPR$0={\"x\":1,\"y\":2.1}\n");
}
{code}
We will get:
{code:java}
org.codehaus.commons.compiler.CompileException: Line 22, Column 124: No
applicable constructor/method found for actual parameters "int,
java.math.BigDecimal"; candidates are: "public static
org.apache.calcite.runtime.GeoFunctions$Geom
org.apache.calcite.runtime.GeoFunctions.ST_MakePoint(java.math.BigDecimal,
java.math.BigDecimal, java.math.BigDecimal)", "public static
org.apache.calcite.runtime.GeoFunctions$Geom
org.apache.calcite.runtime.GeoFunctions.ST_MakePoint(java.math.BigDecimal,
java.math.BigDecimal)"
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
at
org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:9263)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9123)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9025)
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5062)
at org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215){code}
(3)For some functions, it is too late to fail the query in runtime phase.
For example: _*RAND_INTEGER*_ adopts _*OperandTypes.or(OperandTypes.NUMERIC,
OperandTypes.NUMERIC_NUMERIC)*_
{code:java}
@Test public void test2() {
final String sql = "SELECT rand_integer(1.1, 2)";
CalciteAssert.that()
.query(sql)
.planContains("xyxyx")
.returns("EXPR$0={\"x\":1,\"y\":2.1}\n");
}{code}
We will get:
{code:java}
org.codehaus.commons.compiler.CompileException: Line 22, Column 100: No
applicable constructor/method found for actual parameters
"java.math.BigDecimal, int"; candidates are: "public int
org.apache.calcite.runtime.RandomFunction.randIntegerSeed(int, int)"
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
at
org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:9263)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9123){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3561) Support using unnest in Interpreter
Wang Yanlin created CALCITE-3561:
Summary: Support using unnest in Interpreter
Key: CALCITE-3561
URL: https://issues.apache.org/jira/browse/CALCITE-3561
Project: Calcite
Issue Type: Improvement
Reporter: Wang Yanlin
Using unnest in Interpreter will cause exception,
{code:java}
// InterpreterTest
@Test public void testInterpretUnnest() throws Exception {
final String sql = "select * from unnest(multiset[1, 2])";
sql(sql).returnsRows("[1]", "[2]");
}
{code}
got
{code:java}
java.lang.AssertionError: interpreter: no implementation for class
org.apache.calcite.rel.core.Uncollect
at
org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:463)
at
org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:44)
at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72)
at
org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:450)
at
org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:44)
at
org.apache.calcite.interpreter.Interpreter$CompilerImpl.visitRoot(Interpreter.java:408)
at
org.apache.calcite.interpreter.Interpreter.(Interpreter.java:89)
at
org.apache.calcite.test.InterpreterTest$Sql.returnsRows(InterpreterTest.java:121)
at
org.apache.calcite.test.InterpreterTest$Sql.returnsRows(InterpreterTest.java:105)
at
org.apache.calcite.test.InterpreterTest.testInterpretUnnest(InterpreterTest.java:446)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{code}
Add support to use unnest in Interpreter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3560) Allow generic data source (eg. InputStream) in CSVAdapter or FileAdapter(s)
Andrei Sereda created CALCITE-3560: -- Summary: Allow generic data source (eg. InputStream) in CSVAdapter or FileAdapter(s) Key: CALCITE-3560 URL: https://issues.apache.org/jira/browse/CALCITE-3560 Project: Calcite Issue Type: Improvement Components: csv-adapter, file-adapter Affects Versions: 1.21.0 Reporter: Andrei Sereda Assignee: Andrei Sereda Currently CSV Adapter requires data to be stored on disk or remotely because table constructor argument is [URL|https://docs.oracle.com/javase/8/docs/api/java/net/URL.html]. Change CSV and File adapter to accept more generic source like [Readable|https://docs.oracle.com/javase/8/docs/api/java/lang/Readable.html] or [InputStream|https://docs.oracle.com/javase/9/docs/api/java/io/InputStream.html] so adapters can operate on in-memory elements like String. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: JUnit 4 and JUnit 5 annotations do not mix well
+1 on merging #1621 soon. It's a large change so could easily occur code conflict. -Rui On Tue, Dec 3, 2019 at 10:42 AM Andrei Sereda wrote: > Thanks Vladimir. Most of the changes look find to me (left a small comment > in PR) > > Once merged, I'll take care of remaining JUnit4 rules for > Cassandra/ElasticSearch/Mongo etc. > > > On Tue, Dec 3, 2019 at 1:23 PM Vladimir Sitnikov < > sitnikov.vladi...@gmail.com> wrote: > > > Ok, I think PR1621 is ready to go: > > https://github.com/apache/calcite/pull/1621 > > I'm inclined to merge it shortly > > > > The PR converts most of the tests, and by default, it allows JUnit5 only. > > > > Certain modules still have "non-trivial" JUnit4 code, and there was an > > agreement to convert them later. > > The modules receive junit4 dependency via junit4=true in > gradle.properties. > > > > --- > > > > The most "controversial" change from my point of view is the removal of > > *Suite* classes. > > In other words, I have dropped classes like > > CalciteSuite, FileSuite, PlusSuite, and so on. > > > > I guess it is OK, as both Gradle and JUnit5 provide lots of ways to > launch > > tests (e.g. tags are good). > > The good thing about the removal is it would reduce the number of > > merge-conflicts, > > and it would simplify adding new test classes. > > > > --- > > > > The most notable difference between junit4 and junit5 .assert* is that > > junit4 has (message, expected, actual) > > argument order while junit5 uses (expected, actual, message). > > The latter is better when message is big (e.g. concatenated), and junit5 > > allows to use lambda, > > so the message is built only when assertion fails. > > > > I've performed argument shuffling with IntelliJ IDEA's "structural search > > and replace", so it should be OK. > > > > Vladimir > > >
Re: JUnit 4 and JUnit 5 annotations do not mix well
Thanks Vladimir. Most of the changes look find to me (left a small comment in PR) Once merged, I'll take care of remaining JUnit4 rules for Cassandra/ElasticSearch/Mongo etc. On Tue, Dec 3, 2019 at 1:23 PM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: > Ok, I think PR1621 is ready to go: > https://github.com/apache/calcite/pull/1621 > I'm inclined to merge it shortly > > The PR converts most of the tests, and by default, it allows JUnit5 only. > > Certain modules still have "non-trivial" JUnit4 code, and there was an > agreement to convert them later. > The modules receive junit4 dependency via junit4=true in gradle.properties. > > --- > > The most "controversial" change from my point of view is the removal of > *Suite* classes. > In other words, I have dropped classes like > CalciteSuite, FileSuite, PlusSuite, and so on. > > I guess it is OK, as both Gradle and JUnit5 provide lots of ways to launch > tests (e.g. tags are good). > The good thing about the removal is it would reduce the number of > merge-conflicts, > and it would simplify adding new test classes. > > --- > > The most notable difference between junit4 and junit5 .assert* is that > junit4 has (message, expected, actual) > argument order while junit5 uses (expected, actual, message). > The latter is better when message is big (e.g. concatenated), and junit5 > allows to use lambda, > so the message is built only when assertion fails. > > I've performed argument shuffling with IntelliJ IDEA's "structural search > and replace", so it should be OK. > > Vladimir >
[jira] [Created] (CALCITE-3559) Drop HydromaticFileSetCheck from Checkstyle configuration
Vladimir Sitnikov created CALCITE-3559: -- Summary: Drop HydromaticFileSetCheck from Checkstyle configuration Key: CALCITE-3559 URL: https://issues.apache.org/jira/browse/CALCITE-3559 Project: Calcite Issue Type: Task Components: build Affects Versions: 1.21.0 Environment: HydromaticFileSetCheck is not customizable, and it is not supported. HydromaticFileSetCheck is not compatible with Checkstyle 8.x which resolves concurrency issues. Lots of checks implemented in HydromaticFileSetCheck can be automated (==the software could automatically fix the code rather than complain). So we should drop HydromaticFileSetCheck Reporter: Vladimir Sitnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3558) Columns reordering doesn't work in SplunkPushDownRule
Shawn Chen created CALCITE-3558: --- Summary: Columns reordering doesn't work in SplunkPushDownRule Key: CALCITE-3558 URL: https://issues.apache.org/jira/browse/CALCITE-3558 Project: Calcite Issue Type: Bug Components: splunk Affects Versions: 1.21.0 Environment: Splunk 7.3+ Reporter: Shawn Chen Assignee: Shawn Chen Fix For: next The indexing variable i doesn't increase per loop between line 233 to 246 in SplunkPushDownRule.java. This bug causes broken column reordering function. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3557) ClassCastException for using nested multiset or array inside multiset
Wang Yanlin created CALCITE-3557:
Summary: ClassCastException for using nested multiset or array
inside multiset
Key: CALCITE-3557
URL: https://issues.apache.org/jira/browse/CALCITE-3557
Project: Calcite
Issue Type: Bug
Reporter: Wang Yanlin
{code:java}
// JdbcTest
@Test public void testNestedMultiset() {
CalciteAssert.that()
.query("select multiset[ARRAY[1, 2], ARRAY[3, 4]]")
.returns("EXPR$0=[[1, 2], [3, 4]]\n");
CalciteAssert.that()
.query("select multiset[multiset[1, 2], multiset[3, 4]]")
.returns("EXPR$0=[[1, 2], [3, 4]]\n");
}
{code}
Got
{code:java}
java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to
java.lang.Integer
at
org.apache.calcite.avatica.util.AbstractCursor$IntAccessor.getInt(AbstractCursor.java:541)
at
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1318)
at
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1299)
at
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1352)
at
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1326)
at
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1299)
at
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1352)
at org.apache.calcite.avatica.AvaticaSite.get(AvaticaSite.java:305)
at
org.apache.calcite.avatica.AvaticaResultSet.getObject(AvaticaResultSet.java:393)
at
org.apache.calcite.test.CalciteAssert$ResultSetFormatter.rowToString(CalciteAssert.java:1978)
at
org.apache.calcite.test.CalciteAssert$ResultSetFormatter.resultSet(CalciteAssert.java:1966)
at
org.apache.calcite.test.CalciteAssert.lambda$checkResult$2(CalciteAssert.java:303)
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:544)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1514)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1446)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1512)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1495)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1458)
at
org.apache.calcite.test.JdbcTest.testNestedMultiset(JdbcTest.java:2057)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at
com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
Suppressed: org.apache.calcite.util.TestUtil$ExtraInformation: With
materializationsEnabled=false, limit=0, sql=select multiset[multiset[1, 2],
multiset[3, 4]]
at org.apache.calcite.util.TestUtil.rethrow(TestUtil.java:268)
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:560)
... 28 more
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3556) Handling of TO_NUMBER method
vaibhav jain created CALCITE-3556: - Summary: Handling of TO_NUMBER method Key: CALCITE-3556 URL: https://issues.apache.org/jira/browse/CALCITE-3556 Project: Calcite Issue Type: New Feature Reporter: vaibhav jain Handling of TO_NUMBER method. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: JUnit 4 and JUnit 5 annotations do not mix well
Thanks Andrei and Vladimir for pushing this forward. I'm +1 for complete migration to JUnit 5. Having both may lead into problems as the one mentioned in the beginning of this thread. More general using two versions of the same library at the same time usually leads to problems. I checked the code and it seems that we have ~11 uses of @RunWith and ~3 uses of @Rule so it should be feasible to migrate all of them quite fast. I already migrated a couple of parameterized tests to JUnit 5 so I can help if needed. On Mon, Dec 2, 2019 at 11:53 PM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: > Andrei>Perhaps split it into two parts: > > That makes sense. > parameterized and classrules seem to be not that trivial to migrate. > > Then I would revert several changes in > https://github.com/apache/calcite/pull/1621 and it would become a > committable PR. > > Vladimir >
[jira] [Created] (CALCITE-3555) Remove EnumerableDefaults in favor of default implementations of interface methods
Ruben Q L created CALCITE-3555: -- Summary: Remove EnumerableDefaults in favor of default implementations of interface methods Key: CALCITE-3555 URL: https://issues.apache.org/jira/browse/CALCITE-3555 Project: Calcite Issue Type: Task Reporter: Ruben Q L EnumerableDefaults exists because a java before JDK 8 did not allow default implementations of interface methods. Now that has changed we could consider removing EnumerableDefaults. -- This message was sent by Atlassian Jira (v8.3.4#803005)
