[jira] [Created] (CALCITE-6502) Parser loses position information for Expression3
Mihai Budiu created CALCITE-6502:
Summary: Parser loses position information for Expression3
Key: CALCITE-6502
URL: https://issues.apache.org/jira/browse/CALCITE-6502
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.37.0
Reporter: Mihai Budiu
Parser.jj features the following code snipped in the Expression3 production:
{code:java}
if (list1.size() == 1) {
// interpret as single value or query
return list1.get(0);
}
{code}
This is problematic because the span of list1 is not the same as the span of
its first element.
This makes it look like the expression parsed does not include the opening and
closing parens.
This can be a problem for error reporting tools (but there are other use cases
that may be impacted).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6496) Enable tests from outer.iq
Mihai Budiu created CALCITE-6496:
Summary: Enable tests from outer.iq
Key: CALCITE-6496
URL: https://issues.apache.org/jira/browse/CALCITE-6496
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.37.0
Reporter: Mihai Budiu
Many tests in outer.iq are disabled with this comment:
{code}
# The following test is disabled, because we cannot handle non-equi-join.
# Following it are the results from MySQL.
!if (false) {
{code}
Most of these tests could probably be enabled, but they require minor
formatting.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6464) Type inference for DECIMAL division seems incorrect
Mihai Budiu created CALCITE-6464: Summary: Type inference for DECIMAL division seems incorrect Key: CALCITE-6464 URL: https://issues.apache.org/jira/browse/CALCITE-6464 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.37.0 Reporter: Mihai Budiu This bug surfaces if one uses a custom type system, e.g., where DECIMAL is limited to (28, 10). The problem is in RelDataTypeSystem.deriveDecimalDivideType. The JavaDoc of this function gives the algorithm for deriving the division result type. According to these rules, if you divide two numbers of type DECIMAL(28, 10), you should get a result with type DECIMAL(28, 10). But the actual implementation infers a type of DECIMAL(28, 0), which seems incorrect. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6442) Validator rejects FILTER in OVER windows
Mihai Budiu created CALCITE-6442:
Summary: Validator rejects FILTER in OVER windows
Key: CALCITE-6442
URL: https://issues.apache.org/jira/browse/CALCITE-6442
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.37.0
Reporter: Mihai Budiu
The Calcite grammar about windows says this:
{code}
windowedAggregateCall:
agg '(' [ ALL | DISTINCT ] value [, value ]* ')'
[ RESPECT NULLS | IGNORE NULLS ]
[ WITHIN GROUP '(' ORDER BY orderItem [, orderItem ]* ')' ]
[ FILTER '(' WHERE condition ')' ]
OVER window
{code}
However, the validator rejects the following query:
{code:sql}
SELECT deptno,
COUNT(DISTINCT deptno) FILTER (WHERE deptno > 10)
OVER win AS agg
FROM emp
WINDOW win AS (PARTITION BY empno)
{code}
with the following error:
{code}
org.apache.calcite.sql.validate.SqlValidatorException: OVER must be applied to
aggregate function
at
java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
at
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
at
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:484)
at
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:507)
at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:601)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:948)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:933)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:5676)
at
org.apache.calcite.sql.SqlOverOperator.validateCall(SqlOverOperator.java:77)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:6404)
{code}
The easy fix would be to remove this from the grammar.
A harder fix would be to implement support for FILTERs in windows, but I don't
know how hard that would be.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6427) Use a higher precision for DECIMAL intermediate results for some aggregate functions like STDDEV
Mihai Budiu created CALCITE-6427: Summary: Use a higher precision for DECIMAL intermediate results for some aggregate functions like STDDEV Key: CALCITE-6427 URL: https://issues.apache.org/jira/browse/CALCITE-6427 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.37.0 Reporter: Mihai Budiu Today aggregates like STDDEV are expanded into computations that use intermediate results with the same type as the input values. Since STDDEV squares values, this precision may prove insufficient. For DECIMAL values a reasonable approach would be to use double the input precision and scale. This is also related to [CALCITE-6322], [CALCITE-4924], [CALCITE-6324] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6407) DECIMAL types with scale > precision should be tested
Mihai Budiu created CALCITE-6407: Summary: DECIMAL types with scale > precision should be tested Key: CALCITE-6407 URL: https://issues.apache.org/jira/browse/CALCITE-6407 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.37.0 Reporter: Mihai Budiu Calcite allows DECIMAL types where the scale > precision. However, whether these are legal should be probably defined by the type system. This case should be also more thoroughly tested. See also https://issues.apache.org/jira/browse/CALCITE-5651 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6406) Negative scales for DECIMAL types are not tested
Mihai Budiu created CALCITE-6406: Summary: Negative scales for DECIMAL types are not tested Key: CALCITE-6406 URL: https://issues.apache.org/jira/browse/CALCITE-6406 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.37.0 Reporter: Mihai Budiu Calcite allows DECIMAL types to have negative scales. This is in line with other SQL dialects. I suspect that the type system should decide whether negative scales are allowed or not. Moreover, if they are allowed, they should be tested properly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6403) Rule AGGREGATE_EXPAND_DISTINCT_AGGREGATES is unsound
Mihai Budiu created CALCITE-6403:
Summary: Rule AGGREGATE_EXPAND_DISTINCT_AGGREGATES is unsound
Key: CALCITE-6403
URL: https://issues.apache.org/jira/browse/CALCITE-6403
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.37.0
Reporter: Mihai Budiu
Here is a simple SQL program that operates on a collection with no input rows:
{code:sql}
SELECT COUNT(*), COUNT(DISTINCT COL1) FROM T WHERE false"
{code}
The rewrite rule expands this into:
{code}
LogicalProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]), id = 39
LogicalAggregate(group=[{}], EXPR$0=[MIN($1) FILTER $3],
EXPR$1=[COUNT($0) FILTER $2]), id = 37
LogicalProject(COL1=[$0], EXPR$0=[$1], $g_0=[=($2, 0)], $g_1=[=($2,
1)]), id = 35
LogicalAggregate(group=[{0}], groups=[[{0}, {}]], EXPR$0=[COUNT()],
$g=[GROUPING($0)]), id = 30
LogicalValues(tuples=[[]]), id = 22
{code}
Notice that there is an inner group-by aggregate that produces an empty set,
and an outer aggregation that uses MIN. MIN for an empty collection is NULL,
whereas the original query should have produced 0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6395) Significant precision loss when representing REAL literals
Mihai Budiu created CALCITE-6395:
Summary: Significant precision loss when representing REAL literals
Key: CALCITE-6395
URL: https://issues.apache.org/jira/browse/CALCITE-6395
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
Consider this test that could be a SqlOperatorTest:
{code:java}
f.checkScalar("CAST(CAST('36854775807.0' AS REAL) AS BIGINT)",
"36854775808", "BIGINT NOT NULL");
{code}
The produced result is actually very far:
Expected: is "36854775808"
but: was "36854779904"
This big error comes from two reasons:
- Calcite uses BigDecimal values to represent floating point values, see
[CALCITE-2067]
- When converting a Float value to a BigDecimal in RexBuilder.clean(), the
following sequence is used:
{code:java}
new BigDecimal(((Number) o).doubleValue(), MathContext.DECIMAL32)
{code}
Using a DECIMAL32 math context leads to the precision loss. Just because a
Float uses 32 bits does not mean that the decimal should also use 32 bits.
The real fix is in the PR for [CALCITE-2067], but that hasn't been reviewed for
a long time, so I will submit a fix for the clean() method..
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6389) RexBuilder.removeCastFromLiteral does not preserve semantics for decimal literals
Mihai Budiu created CALCITE-6389:
Summary: RexBuilder.removeCastFromLiteral does not preserve
semantics for decimal literals
Key: CALCITE-6389
URL: https://issues.apache.org/jira/browse/CALCITE-6389
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
This is a bug which I keep fixing as part of several pull requests (not yet
merged), so I have decided to make it into a separate issue.
The code in removeCastFromLiteral is supposed to remove casts which are useless.
However, the code is too aggressive for decimal literals. In particular, this
fragment:
{code:java}
if (toType.getSqlTypeName() == SqlTypeName.DECIMAL) {
final BigDecimal decimalValue = (BigDecimal) value;
return SqlTypeUtil.isValidDecimalValue(decimalValue, toType);
}
{code}
There are at least two bugs here:
- this code removes casts from literals even if they represent types such as
interval
- this code does not check properly that the literal can be represented by the
type specified by the cast. In particular, the function
SqlTypeUtil.isValidDecimalValue does not correctly consider the scale of the
resulting type, and may return 'true' even when a number requires rescaling.
Removing casts in such a case changes the literal's value. I have submitted a
fix for this bug as part of [CALCITE-6322], but that PR hasn't been merged yet.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6383) The class SameOperandTypeChecker is incorrectly named and documented
Mihai Budiu created CALCITE-6383:
Summary: The class SameOperandTypeChecker is incorrectly named and
documented
Key: CALCITE-6383
URL: https://issues.apache.org/jira/browse/CALCITE-6383
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The SameOperandTypeChecker claims that it checks whether operands have the same
type (the class name suggests this, as does the JavaDoc).
{code:java}
/**
* Parameter type-checking strategy where all operand types must be the same.
*/
public class SameOperandTypeChecker implements SqlSingleOperandTypeChecker {
{code}
But the code does something this:
{code:java}
for (int i : operandList) {
if (prev >= 0) {
if (!SqlTypeUtil.isComparable(types[i], types[prev])) {
{code}
The documentation for isComparable says:
{code:java}
/**
* Returns whether two types are comparable. They need to be scalar types of
* the same family, or struct types whose fields are pairwise comparable.
{code}
Thus the class only checks that the operands have the same type *family*, not
the same *type*.
I am not sure what the right fix is, though, since changing the class name
would be a pretty big breaking change. But I suspect this confusion is a source
of a few bugs. An instance is [CALCITE-6382]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6382) Type inference for SqlLeadLagAggFunction is incorrect
Mihai Budiu created CALCITE-6382: Summary: Type inference for SqlLeadLagAggFunction is incorrect Key: CALCITE-6382 URL: https://issues.apache.org/jira/browse/CALCITE-6382 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.36.0 Reporter: Mihai Budiu Currently the LeadLag operator does not use the default value when inferring the type of a column. For the following example: ``` SELECT lead(c * 2, 1, -1.4) OVER (PARTITION BY x ORDER BY c) FROM t ``` The column 'c' has type INTEGER, and Calcite infers a type of INTEGER for the result. However, the default value for the lead is -1.4, which is DECIMAL, so the result type should be DECIMAL. Currently Calcite only uses the nullability of the default value, but not its type in the result type inference. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6380) Casts from INTERVAL and STRING to DECIMAL are incorrect
Mihai Budiu created CALCITE-6380: Summary: Casts from INTERVAL and STRING to DECIMAL are incorrect Key: CALCITE-6380 URL: https://issues.apache.org/jira/browse/CALCITE-6380 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.36.0 Reporter: Mihai Budiu This is a follow-up from [CALCITE-6322], which is about casts from numeric types to DECIMAL values. There are two tests in SqlOperatorTest which are disabled due to this bug: - testCastStringToDecimal - testCastIntervalToNumeric -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6379) Arithmetic in Calcite does not produce results with the expected type
Mihai Budiu created CALCITE-6379: Summary: Arithmetic in Calcite does not produce results with the expected type Key: CALCITE-6379 URL: https://issues.apache.org/jira/browse/CALCITE-6379 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.36.0 Reporter: Mihai Budiu This seems to be a very old bug in Calcite, which appears in SqlOperatorTest under the name FNL25. It has been around for at least 10 years, but it affects other parts of the code as well. I could not find a more precise description of this bug in the JIRA, so I am filing a new one. I think this is at least a critical bug. The problem affects the expression evaluator in Calcite. After evaluating an arithmetic operation the result should be cast to the expected result type. For example, adding two SMALLINT values should cast the result to SMALLINT. Currently the code generated by the RexToLixTranslator is missing this cast. Since Java evaluation rules are different from SQL, many results are incorrect. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6377) Time expression causes IllegalStateException
Mihai Budiu created CALCITE-6377:
Summary: Time expression causes IllegalStateException
Key: CALCITE-6377
URL: https://issues.apache.org/jira/browse/CALCITE-6377
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following SqlOperatorTest causes an exception:
{code:java}
final SqlOperatorFixture f = fixture();
f.checkScalar("time '12:03:01' + interval '25' day",
"12:03:01", "TIME(0) NOT NULL");
{code}
The exception is:
{code}
Caused by: java.lang.IllegalStateException: Unable to implement
EnumerableCalc(expr#0=[{inputs}], expr#1=[12:03:01],
expr#2=[216000:INTERVAL DAY], expr#3=[+($t1, $t2)], EXPR$0=[$t3]): rowcount
= 1.0, cumulative cost = {2.0 rows, 6.0 cpu, 0.0 io}, id = 20
EnumerableValues(tuples=[[{ 0 }]]): rowcount = 1.0, cumulative cost = {1.0
rows, 1.0 cpu, 0.0 io}, id = 13
at
org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:117)
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:112)
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1171)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:326)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:666)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:519)
...
Suppressed: java.lang.ArithmeticException: Value 216000 out of range
at
org.apache.calcite.linq4j.tree.Primitive.checkRoundedRange(Primitive.java:383)
at
org.apache.calcite.linq4j.tree.Primitive.numberValue(Primitive.java:412)
at
org.apache.calcite.linq4j.tree.Expressions.constant(Expressions.java:575)
at
org.apache.calcite.linq4j.tree.OptimizeShuttle.visit(OptimizeShuttle.java:305)
at
org.apache.calcite.linq4j.tree.UnaryExpression.accept(UnaryExpression.java:39)
at
org.apache.calcite.linq4j.tree.BinaryExpression.accept(BinaryExpression.java:47)
{code}
This seems to happen because the implementation insists in evaluating the
expression by converting the 25 days interval to milliseconds, which overflows.
However, adding a days interval to a time should be a noop. Replacing 'days'
with 'months', for example, works fine.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6372) Support ASOF joins
Mihai Budiu created CALCITE-6372: Summary: Support ASOF joins Key: CALCITE-6372 URL: https://issues.apache.org/jira/browse/CALCITE-6372 Project: Calcite Issue Type: New Feature Components: core Affects Versions: 1.36.0 Reporter: Mihai Budiu Seems that this new kind of JOIN named AS OF is very useful for processing time-series data. Here is some example documentation from Snowflake: https://docs.snowflake.com/en/sql-reference/constructs/asof-join The semantics is similar to a traditional join, but the result always contains at most one record from the left side, with the last matching record on the right side (where "time" is any value that can be compared for inequality). This can be expressed in SQL, but it looks very cumbersome, using a JOIN, a GROUP BY, and then an aggregation to keep the last value. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6361) Uncollect.deriveUncollectRowType crashes if the input data is not a collection
Mihai Budiu created CALCITE-6361: Summary: Uncollect.deriveUncollectRowType crashes if the input data is not a collection Key: CALCITE-6361 URL: https://issues.apache.org/jira/browse/CALCITE-6361 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.37.0 Reporter: Mihai Budiu This happens because the type checker calls getComponentType() without checking first that the field type has components. It should report an error in such a case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6353) Optimization CoreRules.PROJECT_REDUCE_EXPRESSIONS crashes while optimizing ARRAY_CONCAT expression
Mihai Budiu created CALCITE-6353:
Summary: Optimization CoreRules.PROJECT_REDUCE_EXPRESSIONS crashes
while optimizing ARRAY_CONCAT expression
Key: CALCITE-6353
URL: https://issues.apache.org/jira/browse/CALCITE-6353
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following RelOptRulesTest
{code:java}
@Test void testArrayConcat() {
final String sql = "select array_concat(ARRAY [1, 2], ARRAY [3, 4])";
sql(sql).withFactory(
t -> t.withOperatorTable(
opTab ->
SqlLibraryOperatorTableFactory.INSTANCE.getOperatorTable(
SqlLibrary.STANDARD, SqlLibrary.BIG_QUERY)))
.withRule(CoreRules.PROJECT_REDUCE_EXPRESSIONS)
.check();
}
{code}
crashes with the following stack trace:
{code:java}
java.lang.RuntimeException: While compiling [ARRAY_CONCAT(ARRAY(1, 2), ARRAY(3,
4))]
at org.apache.calcite.rex.RexExecutable.compile(RexExecutable.java:73)
at org.apache.calcite.rex.RexExecutable.(RexExecutable.java:53)
at
org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:145)
at
org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressionsInternal(ReduceExpressionsRule.java:774)
at
org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressions(ReduceExpressionsRule.java:714)
{code}
It seems that the generated code passed to Janino is invalid:
Line 10, Column 5: Assignment conversion not possible from type
"java.util.ArrayList" to type "java.lang.Object[]"
org.codehaus.commons.compiler.CompileException: Line 10, Column 5: Assignment
conversion not possible from type "java.util.ArrayList" to type
"java.lang.Object[]"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6349) CoreRules.PROJECT_REDUCE_EXPRESSIONS crashes on expression with ARRAY_REPEAT
Mihai Budiu created CALCITE-6349:
Summary: CoreRules.PROJECT_REDUCE_EXPRESSIONS crashes on
expression with ARRAY_REPEAT
Key: CALCITE-6349
URL: https://issues.apache.org/jira/browse/CALCITE-6349
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following test in RelOptRulesTest causes a crash:
{code:java}
@Test void testArrayRepeat() {
final String sql = "select array_repeat(1, null)";
sql(sql)
.withFactory(
t -> t.withOperatorTable(
opTab ->
SqlLibraryOperatorTableFactory.INSTANCE.getOperatorTable(
SqlLibrary.STANDARD, SqlLibrary.SPARK)))
.withRule(CoreRules.PROJECT_REDUCE_EXPRESSIONS)
.check();
}
{code}
The crash is:
{code:java}
java.lang.AssertionError: Cannot add expression of different type to set:
set type is RecordType(INTEGER NOT NULL ARRAY NOT NULL EXPR$0) NOT NULL
expression type is RecordType(INTEGER NOT NULL ARRAY EXPR$0) NOT NULL
set is rel#4:LogicalProject.(input=HepRelVertex#3,exprs=[ARRAY_REPEAT(1,
null:DECIMAL(19, 9))])
expression is LogicalProject(EXPR$0=[null:INTEGER NOT NULL ARRAY])
LogicalValues(tuples=[[{ 0 }]])
Type mismatch:
rowtype of original rel: RecordType(INTEGER NOT NULL ARRAY NOT NULL EXPR$0) NOT
NULL
rowtype of new rel: RecordType(INTEGER NOT NULL ARRAY EXPR$0) NOT NULL
Difference:
EXPR$0: INTEGER NOT NULL ARRAY NOT NULL -> INTEGER NOT NULL ARRAY
at
org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:419)
at
org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:60)
at
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:273)
at
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:288)
at
org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:317)
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6348) ARRAY_OVERLAP with a NULL argument crashes the compiler
Mihai Budiu created CALCITE-6348:
Summary: ARRAY_OVERLAP with a NULL argument crashes the compiler
Key: CALCITE-6348
URL: https://issues.apache.org/jira/browse/CALCITE-6348
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following SqlOperatorTest:
{code:java}
f.checkScalar("arrays_overlap(null, null)", true,
"NULL");
{code}
causes a crash:
{code:java}
at java.base/java.util.Objects.requireNonNull(Objects.java:222)
at
org.apache.calcite.sql.type.SqlTypeTransforms.lambda$static$6(SqlTypeTransforms.java:124)
at
org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:66)
at
org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:534)
at
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:503)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:347)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:231)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6714)
{code}
This issue is similar to [CALCITE-6283]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6347) ARRAY_REPEAT with a string argument causes a compiler crash
Mihai Budiu created CALCITE-6347:
Summary: ARRAY_REPEAT with a string argument causes a compiler
crash
Key: CALCITE-6347
URL: https://issues.apache.org/jira/browse/CALCITE-6347
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following SqlOperatorTest:
{code:java}
f.checkScalar("array_repeat('1', 2)", "['1', '1']",
"CHAR(1) NOT NULL ARRAY NOT NULL");
{code}
causes a compiler error:
{code}
Error while compiling generated Java code:
...
static final String $L4J$C$org_apache_calcite_runtime_SqlFunctions_repeat_1_2_
= org.apache.calcite.runtime.SqlFunctions.repeat("1", 2);
...
at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128)
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1171)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:326)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:666)
{code}
This happens because the "repeat" function in SqlFunctions is overloaded to
implement both ARRAY_REPEAT and REPEAT.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6345) Intervals with more than 100 years are not supported
Mihai Budiu created CALCITE-6345:
Summary: Intervals with more than 100 years are not supported
Key: CALCITE-6345
URL: https://issues.apache.org/jira/browse/CALCITE-6345
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
Adding the following SqlValidatorTest:
{code:java}
expr("INTERVAL '100-2' YEAR TO MONTH").assertInterval(is(122L));
{code}
causes the following exception:
{code}
org.apache.calcite.runtime.CalciteContextException: From line 1, column 9 to
line 1, column 38: Interval field value 100 exceeds precision of YEAR(2) field
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:507)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:948)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:933)
at
org.apache.calcite.sql.SqlIntervalQualifier.fieldExceedsPrecisionException(SqlIntervalQualifier.java:1355)
at
org.apache.calcite.sql.SqlIntervalQualifier.checkLeadFieldInRange(SqlIntervalQualifier.java:475)
at
org.apache.calcite.sql.SqlIntervalQualifier.evaluateIntervalLiteralAsYearToMonth(SqlIntervalQualifier.java:626)
at
org.apache.calcite.sql.SqlIntervalQualifier.evaluateIntervalLiteral(SqlIntervalQualifier.java:1293)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateLiteral(SqlValidatorImpl.java:3429)
{code}
The spec does not limit years to 2 digits, so I don't know where the YEAR(2)
time is coming from.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6335) Quidem tests should allow specifying optimization passes to apply to programs
Mihai Budiu created CALCITE-6335: Summary: Quidem tests should allow specifying optimization passes to apply to programs Key: CALCITE-6335 URL: https://issues.apache.org/jira/browse/CALCITE-6335 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.36.0 Reporter: Mihai Budiu This facility would make it much easier to test various optimization passes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6332) Optimization CoreRules.AGGREGATE_EXPAND_DISTINCT_AGGREGATES_TO_JOIN produces incorrect results for aggregates with groupSets
Mihai Budiu created CALCITE-6332:
Summary: Optimization
CoreRules.AGGREGATE_EXPAND_DISTINCT_AGGREGATES_TO_JOIN produces incorrect
results for aggregates with groupSets
Key: CALCITE-6332
URL: https://issues.apache.org/jira/browse/CALCITE-6332
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The optimization rule does not seem to consider the groupSets at all.
The following two queries produce the same resulting plan:
{code:sql}
select count(distinct deptno) as cd, count(*) as c
from emp
group by cube(deptno)
{code}
{code:sql}
select count(distinct deptno) as cd, count(*) as c
from emp
group by deptno
{code}
(Notice that one query has a cube, while the other one doesn't)
The produced plan is:
{code}
LogicalProject(CD=[$1], C=[$2]), id = 196
LogicalAggregate(group=[{0}], CD=[COUNT($0)], C=[$SUM0($1)]), id = 201
LogicalAggregate(group=[{0}], C=[COUNT()]), id = 198
LogicalProject(DEPTNO=[$8]), id = 192
LogicalTableScan(table=[[schema, EMP]]), id = 163
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6328) The BigQuery functions SAFE_* do not match the BigQuery specification
Mihai Budiu created CALCITE-6328:
Summary: The BigQuery functions SAFE_* do not match the BigQuery
specification
Key: CALCITE-6328
URL: https://issues.apache.org/jira/browse/CALCITE-6328
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The BigQuery dialect does not support DECIMAL values with arbitrary types: it
only supports two fixed types: NUMERIC and BIGNUMERIC, both with fixed
precision and scale.
The runtime implementation of the SAFE_* functions uses the following helper in
SqlFunctions:
{code:java}
/** Returns whether a BigDecimal value is safe (that is, has not overflowed).
* According to BigQuery, BigDecimal overflow occurs if the precision is
greater
* than 76 or the scale is greater than 38. */
private static boolean safeDecimal(BigDecimal b) {
return b.scale() <= 38 && b.precision() <= 76;
}
{code}
This helper does not handle correctly NUMERIC value, only BIGNUMERIC.
Moreover, all the tests in SqlOperatorTests use a type system which doesn't
even support DECIMAL values wider than 38 digits. So a test like the following:
{code:java}
f.checkNull("safe_add(cast(-9.9e75 as DECIMAL(76, 0)), "
+ "cast(-9.9e75 as DECIMAL(76, 0)))");
{code}
cannot even create the expected BigDecimal value correctly.
This surfaced during the attempt to fix [CALCITE-6322]: once the casts to
DECIMAL are implemented, some of these tests break.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6324) Type inferred for result of STDDEV, VAR_SAMP, etc. is incorrect
Mihai Budiu created CALCITE-6324: Summary: Type inferred for result of STDDEV, VAR_SAMP, etc. is incorrect Key: CALCITE-6324 URL: https://issues.apache.org/jira/browse/CALCITE-6324 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.36.0 Reporter: Mihai Budiu These functions are all use the same type inference algorithm, essentially the algorithm used by AVG. But if the values processed are decimal, STDDEV (and others) need much higher precision to represent the result. (I am not sure that the inference is right for integer types either, btw.) This surfaced during the implementation of a fix for [CALCITE-6322]: if we use the type inferred for these functions, the result overflows and causes a runtime exception. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6322) Casts to DECIMAL types are ignored
Mihai Budiu created CALCITE-6322:
Summary: Casts to DECIMAL types are ignored
Key: CALCITE-6322
URL: https://issues.apache.org/jira/browse/CALCITE-6322
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following SqlOperatorTest fails:
{code:java}
f.checkScalar("CAST(1.123 AS DECIMAL(4, 0))", "1.0", "DECIMAL(4, 0) NOT NULL");
{code}
The result computed by Calcite is 1.123, ignoring the scale of the DECIMAL
result.
Spark, Postgres, MySQL all return 1.0.
I have marked this as a major bug.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6317) Optimization CoreRules.PROJECT_REDUCE_EXPRESSIONS is unsound
Mihai Budiu created CALCITE-6317:
Summary: Optimization CoreRules.PROJECT_REDUCE_EXPRESSIONS is
unsound
Key: CALCITE-6317
URL: https://issues.apache.org/jira/browse/CALCITE-6317
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
Here is a query taken from agg.iq:
{code:sql}
select deptno, gender, grouping_id(deptno, gender, deptno), count(*) as c
from emp
where deptno = 10
group by rollup(gender, deptno)
{code}
The query plan initially is
{code}
LogicalProject(DEPTNO=[$1], GENDER=[$0], EXPR$2=[$2], C=[$3]), id = 72
LogicalAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {}]],
EXPR$2=[GROUPING_ID($1, $0, $1)], C=[COUNT()]), id = 71
LogicalProject(GENDER=[$2], DEPTNO=[$1]), id = 70
LogicalFilter(condition=[=($1, 10)]), id = 66
LogicalTableScan(table=[[schema, EMP]]), id = 65
{code}
After applying PROJECT_REDUCE_EXPRESSIONS the plan looks like:
{code}
LogicalProject(DEPTNO=[CAST(10):INTEGER], GENDER=[$0], EXPR$2=[$2],
C=[$3]), id = 82
LogicalAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {}]],
EXPR$2=[GROUPING_ID($1, $0, $1)], C=[COUNT()]), id = 78
LogicalProject(GENDER=[$2], DEPTNO=[CAST(10):INTEGER]), id = 84
LogicalFilter(condition=[=($1, 10)]), id = 74
LogicalTableScan(table=[[schema, EMP]]), id = 65
{code}
The problem is in the outer LogicalProject, where the value 10 has replaced
DEPTNO.
However, DEPTNO can also be NULL, because of the groups in the LogicalAggregate.
The constant should not be pushed past the aggregation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6286) Optimizing ARRAY_INSERT expression causes an assertion failure
Mihai Budiu created CALCITE-6286:
Summary: Optimizing ARRAY_INSERT expression causes an assertion
failure
Key: CALCITE-6286
URL: https://issues.apache.org/jira/browse/CALCITE-6286
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following RelOptRulesTest causes an assertion failure:
{code:java}
@Test void testArrayRepeat() {
sql("SELECT array_repeat(123, null)")
.withFactory(
t -> t.withOperatorTable(
opTab ->
SqlLibraryOperatorTableFactory.INSTANCE.getOperatorTable(
SqlLibrary.STANDARD, SqlLibrary.SPARK)))
.withRule(CoreRules.PROJECT_REDUCE_EXPRESSIONS).checkUnchanged();
}
{code}
The assertion failure is:
{code}
java.lang.AssertionError: Cannot add expression of different type to set:
set type is RecordType(INTEGER NOT NULL ARRAY NOT NULL EXPR$0) NOT NULL
expression type is RecordType(INTEGER NOT NULL ARRAY EXPR$0) NOT NULL
set is rel#4:LogicalProject.(input=HepRelVertex#3,exprs=[ARRAY_REPEAT(123,
null:DECIMAL(19, 9))])
expression is LogicalProject(EXPR$0=[null:INTEGER NOT NULL ARRAY])
LogicalValues(tuples=[[{ 0 }]])
Type mismatch:
rowtype of original rel: RecordType(INTEGER NOT NULL ARRAY NOT NULL EXPR$0) NOT
NULL
rowtype of new rel: RecordType(INTEGER NOT NULL ARRAY EXPR$0) NOT NULL
Difference:
EXPR$0: INTEGER NOT NULL ARRAY NOT NULL -> INTEGER NOT NULL ARRAY
at
org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:419)
at
org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:60)
at
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:273)
at
org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:288)
at
org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:317)
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337)
{code}
This suggests that the type inferred for ARRAY_INSERT is incorrect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6285) Function ARRAY_INSERT produces an incorrect result for negative indices
Mihai Budiu created CALCITE-6285:
Summary: Function ARRAY_INSERT produces an incorrect result for
negative indices
Key: CALCITE-6285
URL: https://issues.apache.org/jira/browse/CALCITE-6285
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
Here is a test taken from the Spark documentation page:
https://spark.apache.org/docs/latest/api/sql/index.html#array_insert
{code}
SELECT array_insert(array(5, 3, 2, 1), -4, 4);
[5,4,3,2,1]
{code}
The result produced by Calcite is:
[4,5,3,2,1]
The strange thing is that there are tests for negative indices. I wonder if the
original tests are wrong, or the behavior of this function in Spark was changed
since the tests were written.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6283) array_append function with a NULL array argument crashes with a NullPointerException
Mihai Budiu created CALCITE-6283:
Summary: array_append function with a NULL array argument crashes
with a NullPointerException
Key: CALCITE-6283
URL: https://issues.apache.org/jira/browse/CALCITE-6283
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following test added to SqlOperatorTest:
{code:java}
f.checkNull("array_append(null, 2)");
{code}
causes Calcite to crash with the following stack trace:
{code}
java.lang.NullPointerException: componentType is null for NULL
at java.base/java.util.Objects.requireNonNull(Objects.java:347)
at
org.apache.calcite.sql.type.NonNullableAccessors.getComponentTypeOrThrow(NonNullableAccessors.java:52)
at
org.apache.calcite.sql.type.ArrayElementOperandTypeChecker.checkOperandTypes(ArrayElementOperandTypeChecker.java:49)
at
org.apache.calcite.sql.SqlOperator.checkOperandTypes(SqlOperator.java:761)
at
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:498)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:347)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:231)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6282) Avatica ignores time precision when returning TIME results
Mihai Budiu created CALCITE-6282:
Summary: Avatica ignores time precision when returning TIME results
Key: CALCITE-6282
URL: https://issues.apache.org/jira/browse/CALCITE-6282
Project: Calcite
Issue Type: Bug
Components: avatica, core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
SqlOperatorTest contains the following disabled test:
{code:java}
f.checkScalar("cast(TIME '12:42:25.34' as TIME(2))",
"12:42:25.34", "TIME(2) NOT NULL");
{code}
This bug is disabled based on the following condition;
{code:java}
/**
* Whether http://issues.eigenbase.org/browse/FRG-282;>issue
* FRG-282: Support precision in TIME and TIMESTAMP data types is fixed.
*/
public static final boolean FRG282_FIXED = true;
{code}
However, the result is computed correctly. The precision is lost in the JDBC
layer, which creates a TimeFromNumberAccessor which does not depend on the
precision of the target type: it always returns the time with a precision of 0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6275) Parser for data types ignores element nullability in collections
Mihai Budiu created CALCITE-6275:
Summary: Parser for data types ignores element nullability in
collections
Key: CALCITE-6275
URL: https://issues.apache.org/jira/browse/CALCITE-6275
Project: Calcite
Issue Type: Bug
Components: core, server
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The parser (Parser.jj) has this production rule for DataType:
{code}
// Type name with optional scale and precision.
SqlDataTypeSpec DataType() :
{
SqlTypeNameSpec typeName;
final Span s;
}
{
typeName = TypeName() {
s = Span.of(typeName.getParserPos());
}
(
typeName = CollectionsTypeName(typeName)
)*
{
return new SqlDataTypeSpec(typeName,
s.add(typeName.getParserPos()).pos());
}
}
{code}
Note that there is no way to specify the nullability for the elements of a
collection, they are always assumed to be non-null. This is most pertinent for
the server component, where in DDL one cannot specify a table column of type
INTEGER ARRAY; one always gets an INTEGER NOT NULL ARRAY instead.
But note that SqlCollectionTypeNameSpec cannot even represent the nullability
of the elements' type, it takes a SqlTypeNameSpec instead of a SqlDataTypeSpec.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6253) BigQuery FORMAT_DATE function does not implement all format specifiers
Mihai Budiu created CALCITE-6253:
Summary: BigQuery FORMAT_DATE function does not implement all
format specifiers
Key: CALCITE-6253
URL: https://issues.apache.org/jira/browse/CALCITE-6253
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following query:
{code:sql}
SELECT
format_date('%A,%a,%B,%b,%C,%D,%d,%e,%F,%G,%g,%h,%j,%m,%U,%u,%V,%W,%w,%x,%Y,%y,%t,%%','2024-02-07')
{code}
produces the following output in Calcite:
{code}
Wednesday,Wed,Feb,Feb,%C,%D,07,07,2024-02-07,%G,%g,%h,038,02,%U,4,06,06,%w,02/07/24,2024,24,%t,%%
{code}
The expected output, checked in the BigQuery playground, is:
{code}
Wednesday,Wed,February,Feb,20,02/07/24,07,
7,2024-02-07,2024,24,Feb,038,02,05,3,06,06,3,02/07/24,2024,24,\t,%
{code}
This implies that the following format specifiers are not implemented: %C, %D,
%G, %g, %h, %U, %w, %t, %%.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6252) BigQuery FORMAT_DATE seems to use the wrong calendar for Julian dates
Mihai Budiu created CALCITE-6252:
Summary: BigQuery FORMAT_DATE seems to use the wrong calendar for
Julian dates
Key: CALCITE-6252
URL: https://issues.apache.org/jira/browse/CALCITE-6252
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
For the following query:
{code:sql}
SELECT format_date('%A %d %B %Y', '0001-02-28')
{code}
the BigQuery playground returns the following result:
{code}
Monday 01 January 1
{code}
However, Calcite returns the following result:
{code}
Saturday 01 Jan 1
{code}
There are actually two bugs here:
- the day of the week is wrong
- the month name is displayed incorrectly. The latter is because of the
Locale.ROOT used in SimpleDateFormat.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6248) Illegal dates are accepted by casts
Mihai Budiu created CALCITE-6248:
Summary: Illegal dates are accepted by casts
Key: CALCITE-6248
URL: https://issues.apache.org/jira/browse/CALCITE-6248
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following test passes in SqlOperatorTest:
{code:java}
@Test public void testIllegalDate() {
final SqlOperatorFixture f = fixture();
f.checkScalar("cast('1945-02-32' as DATE)",
"1945-03-04", "DATE NOT NULL");
}
{code}
There is no February 32, I suspect that this expression should produce an error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6247) BigQuery FORMAT_DATE function handles incorrectly the %e format specifier
Mihai Budiu created CALCITE-6247:
Summary: BigQuery FORMAT_DATE function handles incorrectly the %e
format specifier
Key: CALCITE-6247
URL: https://issues.apache.org/jira/browse/CALCITE-6247
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
According to the BigQuery spec
https://cloud.google.com/bigquery/docs/reference/standard-sql/format-elements#format_elements_date_time
the %e format specified should mean: "The day of month as a decimal number
(1-31); single digits are preceded by a space."
However, the implementation in FormatModels.java BIGQUERY is:
{code:java}
map.put("%e", DD);
{code}
which is the same as %d, which uses a leading 0 instead of a space.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6238) Exception while evaluating ROUND function
Mihai Budiu created CALCITE-6238:
Summary: Exception while evaluating ROUND function
Key: CALCITE-6238
URL: https://issues.apache.org/jira/browse/CALCITE-6238
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The following test in CalciteSqlOperatorTest:
{code:java}
@Test void testRoundFunc() {
final SqlOperatorFixture f = fixture();
f.checkScalar("round(42, CAST(2 as BIGINT))", 42, "INTEGER NOT NULL");
}
{code}
causes an exception; here is the relevant part of the stack trace:
{code}
java.sql.SQLException: Error while executing SQL "values (round(42, CAST(2 as
BIGINT)))": Unable to implement EnumerableCalc(expr#0=[{inputs}], expr#1=[42],
expr#2=[2:BIGINT], expr#3=[ROUND($t1, $t2)], EXPR$0=[$t3]): rowcount = 1.0,
cumulative cost = {2.0 rows, 6.0 cpu, 0.0 io}, id = 20
EnumerableValues(tuples=[[{ 0 }]]): rowcount = 1.0, cumulative cost = {1.0
rows, 1.0 cpu, 0.0 io}, id = 13
...
Suppressed: java.lang.RuntimeException: while resolving method
'sround[int, long]' in class class org.apache.calcite.runtime.SqlFunctions
at
org.apache.calcite.adapter.enumerable.EnumUtils.call(EnumUtils.java:679)
at
org.apache.calcite.adapter.enumerable.RexImpTable$MethodImplementor.call(RexImpTable.java:2818)
at
org.apache.calcite.adapter.enumerable.RexImpTable$MethodImplementor.implementSafe(RexImpTable.java:2799)
at
org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.genValueStatement(RexImpTable.java:3857)
at
org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.implement(RexImpTable.java:3819)
{code}
And indeed, SqlFunctions does not have a function sround with this signature.
There are several possible fixes:
- reject calls to ROUND that have a BIGINT second argument
- have the validator insert an implicit cast for the second argument to INTEGER
- implement more Java versions of the SROUND function in SqlFunctions. Probably
many more.
Which one of these is the right one? I suspect this problem applies to other
SQL functions as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6228) ELEMENT function infers incorrect return type
Mihai Budiu created CALCITE-6228:
Summary: ELEMENT function infers incorrect return type
Key: CALCITE-6228
URL: https://issues.apache.org/jira/browse/CALCITE-6228
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The ELEMENT function is defined in the documentation as follows:
[https://calcite.apache.org/docs/reference.html#collection-functions]
{quote}Returns the sole element of an array or multiset; null if the collection
is empty; throws if it has more than one element.
{quote}
However, the type inference returns just the type of the element of the
collection, without changing its nullability.
The type inference is implemented as follows in SqlStdOperatorTable:
{code:java}
public static final SqlFunction ELEMENT =
SqlBasicFunction.create("ELEMENT",
ReturnTypes.MULTISET_ELEMENT_NULLABLE,
OperandTypes.COLLECTION);
{code}
However, reading the definition of MULTISET_ELEMENT_NULLABLE in
ReturnTypes.java:
{code:java}
public static final SqlReturnTypeInference MULTISET_ELEMENT_NULLABLE =
MULTISET.andThen(SqlTypeTransforms.TO_COLLECTION_ELEMENT_TYPE);
{code}
we notice that it is not forced to be nullable. Probably the correct
implementation would be
{code:java}
public static final SqlReturnTypeInference MULTISET_ELEMENT_NULLABLE =
MULTISET.andThen(SqlTypeTransforms.TO_COLLECTION_ELEMENT_TYPE)
.andThen(SqlTypeTransforms.FORCE_NULLABLE);
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6227) ELEMENT(NULL) causes an assertion failure
Mihai Budiu created CALCITE-6227:
Summary: ELEMENT(NULL) causes an assertion failure
Key: CALCITE-6227
URL: https://issues.apache.org/jira/browse/CALCITE-6227
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
Adding the following test to SqlValidatorTest:
{code:java}
@Test void testElement() {
expr("element(null)").columnType("null");
}
{code}
causes an assertion failure. The bottom of the stack trace is the following:
{code}
at org.apache.calcite.sql.type.ReturnTypes$2.get(ReturnTypes.java:577)
at org.apache.calcite.sql.type.ReturnTypes$2.get(ReturnTypes.java:571)
at
org.apache.calcite.sql.ExplicitOperatorBinding.getOperandType(ExplicitOperatorBinding.java:78)
at
org.apache.calcite.sql.SqlOperatorBinding$1.get(SqlOperatorBinding.java:255)
at
org.apache.calcite.sql.SqlOperatorBinding$1.get(SqlOperatorBinding.java:253)
at
org.apache.calcite.sql.type.SqlTypeFactoryImpl.leastRestrictive(SqlTypeFactoryImpl.java:174)
at
org.apache.calcite.rel.type.RelDataTypeFactory.leastRestrictive(RelDataTypeFactory.java:222)
at
org.apache.calcite.sql.type.ReturnTypes.lambda$static$3(ReturnTypes.java:537)
at
org.apache.calcite.sql.type.ReturnTypes.lambda$static$6(ReturnTypes.java:586)
at
org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:58)
at
org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:534)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6210) Cast to VARBINARY causes an assertion failure
Mihai Budiu created CALCITE-6210:
Summary: Cast to VARBINARY causes an assertion failure
Key: CALCITE-6210
URL: https://issues.apache.org/jira/browse/CALCITE-6210
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
This test in SqlOperatorTest:
{code:java}
SqlOperatorFixture f = fixture();
f.checkScalar("CAST('00' AS VARBINARY)", "00", "VARBINARY NOT NULL");
{code}
Causes the following assertion failure:
{code}
java.lang.AssertionError: value 00 does not match type class
org.apache.calcite.avatica.util.ByteString
at
org.apache.calcite.linq4j.tree.ConstantExpression.(ConstantExpression.java:51)
at
org.apache.calcite.linq4j.tree.Expressions.constant(Expressions.java:585)
at
org.apache.calcite.linq4j.tree.OptimizeShuttle.visit(OptimizeShuttle.java:305)
at
org.apache.calcite.linq4j.tree.UnaryExpression.accept(UnaryExpression.java:39)
at
org.apache.calcite.linq4j.tree.TernaryExpression.accept(TernaryExpression.java:47)
at
org.apache.calcite.linq4j.tree.DeclarationStatement.accept(DeclarationStatement.java:45)
at
org.apache.calcite.linq4j.tree.DeclarationStatement.accept(DeclarationStatement.java:27)
at
org.apache.calcite.linq4j.tree.BlockBuilder.optimize(BlockBuilder.java:426)
at
org.apache.calcite.linq4j.tree.BlockBuilder.toBlock(BlockBuilder.java:340)
at
org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:102)
at
org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:68)
at
org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:133)
at
org.apache.calcite.rex.RexSimplify.simplifyCast(RexSimplify.java:2272)
at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:292)
at
org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:250)
at
org.apache.calcite.rex.RexSimplify.simplifyPreservingType(RexSimplify.java:189)
at
org.apache.calcite.rex.RexSimplify.simplifyPreservingType(RexSimplify.java:184)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6194) Contributor rules do not give instructions about how to quote commits
Mihai Budiu created CALCITE-6194: Summary: Contributor rules do not give instructions about how to quote commits Key: CALCITE-6194 URL: https://issues.apache.org/jira/browse/CALCITE-6194 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.36.0 Reporter: Mihai Budiu https://calcite.apache.org/docs/howto.html#merging-pull-requests According to [~jhyde] (in a comment on [CALCITE-6178] when an issue is closed the commit that is quoted in the JIRA case has to be with respect to the main branch of Calcite. The instructions for committers do not specify this information. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6169) EnumUtils.convert does not implement the correct SQL cast semantics
Mihai Budiu created CALCITE-6169: Summary: EnumUtils.convert does not implement the correct SQL cast semantics Key: CALCITE-6169 URL: https://issues.apache.org/jira/browse/CALCITE-6169 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.36.0 Reporter: Mihai Budiu Assignee: Mihai Budiu The code generated by EnumUtil for casts uses the Java semantics, where casts just truncate values. The SQL semantics requires a runtime exception if the value does not fit in the target datatype. Some of the bugs in this code generator are masked by the BlockBuilder optimizer which attempts to optimize the generated code using the SQL semantics... But if the code is not optimized and is executed in Java it generates wrong results for most conversions that overflow. One additional wrinkle is that it is not entirely clear whether the bug is in EnumUtils or in the code that invokes EnumUtils. It is not specified which of the two semantics for casts EnumUtils are supposed to implement: Java or SQL? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6168) RexExecutor can throw during compilation
Mihai Budiu created CALCITE-6168:
Summary: RexExecutor can throw during compilation
Key: CALCITE-6168
URL: https://issues.apache.org/jira/browse/CALCITE-6168
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The RexExecutor is supposed to evaluate expressions at compilation time; if an
expression causes an exception, it should be caught and the expression should
be unchanged. The goal is to have the exception be reported at runtime. The
executor does catch exceptions during its "execute" phase, but some exceptions
can be caught during its "compile" phase.
The following SqlOperatorTest will exhibit such an instance:
{code:java}
@Test
void testCast() {
final SqlOperatorFixture f = fixture();
f.checkFails("CAST(200 as TINYINT)", "", true);
}
}
{code}
This is the relevant portion of the stack trace:
{code}
at
org.apache.calcite.linq4j.tree.Primitive.checkRoundedRange(Primitive.java:383)
at
org.apache.calcite.linq4j.tree.Primitive.numberValue(Primitive.java:398)
at
org.apache.calcite.linq4j.tree.Expressions.constant(Expressions.java:575)
at
org.apache.calcite.linq4j.tree.OptimizeShuttle.visit(OptimizeShuttle.java:305)
at
org.apache.calcite.linq4j.tree.UnaryExpression.accept(UnaryExpression.java:39)
at
org.apache.calcite.linq4j.tree.TernaryExpression.accept(TernaryExpression.java:47)
at
org.apache.calcite.linq4j.tree.Expressions.acceptExpressions(Expressions.java:3214)
at
org.apache.calcite.linq4j.tree.NewArrayExpression.accept(NewArrayExpression.java:49)
at
org.apache.calcite.linq4j.tree.GotoStatement.accept(GotoStatement.java:64)
at
org.apache.calcite.linq4j.tree.BlockBuilder.optimize(BlockBuilder.java:455)
at
org.apache.calcite.linq4j.tree.BlockBuilder.toBlock(BlockBuilder.java:340)
at
org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:102)
at
org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:68)
at
org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:133)
at
org.apache.calcite.rex.RexSimplify.simplifyCast(RexSimplify.java:2265)
at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:292)
at
org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:250)
at
org.apache.calcite.rex.RexSimplify.simplifyPreservingType(RexSimplify.java:189)
at
org.apache.calcite.rex.RexSimplify.simplifyPreservingType(RexSimplify.java:184)
at
org.apache.calcite.tools.RelBuilder.lambda$project_$7(RelBuilder.java:2050)
{code}
Notice that this happens during the compile stage:
at
org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:68)
The simplest fix is probably to move the try/catch block earlier in the flow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6138) Parser does not accept TIMESTAMP WITH TIME ZONE as a data type
Mihai Budiu created CALCITE-6138: Summary: Parser does not accept TIMESTAMP WITH TIME ZONE as a data type Key: CALCITE-6138 URL: https://issues.apache.org/jira/browse/CALCITE-6138 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.36.0 Reporter: Mihai Budiu The current SQL grammar accepts either TIMESTAMP WITHOUT TIME ZONE or TIMESTAMP WITH LOCAL TIME ZONE. It does not accept the version without LOCAL: TIMESTAMP WITH TIME ZONE. This is a legal type in other SQL dialects, and it appears in the SQL 92 standard. The Calcite documentation also lists the rejected type as a legal scalar type: [https://calcite.apache.org/docs/reference.html#scalar-types] So there is a bug: either the compiler is wrong, or the documentation is wrong. The main question is "which one?" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6129) SqlToRelConverter throws an exception when converting a NULL SqlLiteral
Mihai Budiu created CALCITE-6129:
Summary: SqlToRelConverter throws an exception when converting a
NULL SqlLiteral
Key: CALCITE-6129
URL: https://issues.apache.org/jira/browse/CALCITE-6129
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu
The problem is in SqlNodeToRexConverterImpl.convertLiteral. This function
attempts to handle NULL literals differently from all other literals:
{code:java}
final RexBuilder rexBuilder = cx.getRexBuilder();
if (literal.getValue() == null) {
RelDataType type = cx.getValidator().getValidatedNodeType(literal);
return rexBuilder.makeNullLiteral(type);
}
switch (literal.getTypeName()) {
...
{code}
However, such a literal does *not* have a validated data type, to
getValidatedNodeType will throw Util.needToImplement.
The solution would be to handle NULL literals like all other literals and
create a literal with a NULL type.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6105) Documentation does not specify the behavior of SPLIT function for empty string arguments
Mihai Budiu created CALCITE-6105: Summary: Documentation does not specify the behavior of SPLIT function for empty string arguments Key: CALCITE-6105 URL: https://issues.apache.org/jira/browse/CALCITE-6105 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.36.0 Reporter: Mihai Budiu Various string libraries choose different behaviors for SPLIT when the first or the second arguments are empty strings. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6095) Arithmetic expression with VARBINARY value causes AssertionFailure
Mihai Budiu created CALCITE-6095:
Summary: Arithmetic expression with VARBINARY value causes
AssertionFailure
Key: CALCITE-6095
URL: https://issues.apache.org/jira/browse/CALCITE-6095
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following SqlOperatorTest causes an AssertionFailure:
{code:java}
f.check("SELECT x'31' + 0", "INTEGER NOT NULL", "50");
{code}
The top of the stack trace is:
{code}
at
org.apache.calcite.sql.validate.implicit.AbstractTypeCoercion.needToCast(AbstractTypeCoercion.java:297)
at
org.apache.calcite.sql.validate.implicit.AbstractTypeCoercion.coerceOperandType(AbstractTypeCoercion.java:117)
at
org.apache.calcite.sql.validate.implicit.TypeCoercionImpl.binaryArithmeticWithStrings(TypeCoercionImpl.java:200)
at
org.apache.calcite.sql.validate.implicit.TypeCoercionImpl.binaryArithmeticCoercion(TypeCoercionImpl.java:172)
at
org.apache.calcite.sql.type.CompositeOperandTypeChecker.checkOperandTypes(CompositeOperandTypeChecker.java:261)
at
org.apache.calcite.sql.SqlOperator.checkOperandTypes(SqlOperator.java:761)
at
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:498)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:607)
at
org.apache.calcite.sql.SqlBinaryOperator.deriveType(SqlBinaryOperator.java:178)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6094) lin4j.ConstantExpression.write crashes on special FP values
Mihai Budiu created CALCITE-6094:
Summary: lin4j.ConstantExpression.write crashes on special FP
values
Key: CALCITE-6094
URL: https://issues.apache.org/jira/browse/CALCITE-6094
Project: Calcite
Issue Type: Bug
Components: linq4j
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following SqlOperatorTest
{code:java}
f.check("SELECT CAST(10e70 AS REAL)", "REAL NOT NULL", "Infinity");
{code}
crashes with the following top of stack trace:
{code}
Caused by: java.lang.NumberFormatException: Character I is neither a decimal
digit number, decimal point, nor "e" notation exponential mark.
at java.base/java.math.BigDecimal.(BigDecimal.java:522)
at java.base/java.math.BigDecimal.(BigDecimal.java:405)
at java.base/java.math.BigDecimal.(BigDecimal.java:838)
at java.base/java.math.BigDecimal.valueOf(BigDecimal.java:1318)
at
org.apache.calcite.linq4j.tree.ConstantExpression.write(ConstantExpression.java:109)
at
org.apache.calcite.linq4j.tree.ConstantExpression.accept(ConstantExpression.java:77)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6078) Explicit cast to DECIMAL types do not check for overflow
Mihai Budiu created CALCITE-6078: Summary: Explicit cast to DECIMAL types do not check for overflow Key: CALCITE-6078 URL: https://issues.apache.org/jira/browse/CALCITE-6078 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.35.0 Reporter: Mihai Budiu This is a follow-up from [CALCITE-5990] That issue dealt with integers and floating point. This issue is about casts to DECIMAL in which the cast value exceeds the scale of the target result. Apparently Calcite does not handle such casts properly. There are multiple disabled SqlOperatorTests for this condition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6074) The size of REAL, DOUBLE, and FLOAT is not consistent
Mihai Budiu created CALCITE-6074:
Summary: The size of REAL, DOUBLE, and FLOAT is not consistent
Key: CALCITE-6074
URL: https://issues.apache.org/jira/browse/CALCITE-6074
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
This stems from the review of CALCITE-6052
Which one is 8 bytes and which one is 4 bytes?
The intent seems to be that DOUBLE and FLOAT are synonyms, both using 8 bytes,
(which is very weird for Java users), and REAL is 4 bytes.
But an audit of the code shows that:
In AggregateNode.maxMinClass:
{code:java}
case FLOAT:
return max ? MaxFloat.class : MinFloat.class;
case DOUBLE:
case REAL:
return max ? MaxDouble.class : MinDouble.class;
{code}
In VisitorDataContext:
{code:java}
case DOUBLE:
return Pair.of(index, rexLiteral.getValueAs(Double.class));
case REAL:
return Pair.of(index, rexLiteral.getValueAs(Float.class));
{code}
(no case for FLOAT)
In RelMdSize:
{code:java}
case FLOAT:
case REAL:
return 4d;
{code}
in RelDataTypeFactoryImpl:
{code:java}
case REAL:
return createSqlType(SqlTypeName.DECIMAL, 14, 7);
case FLOAT:
return createSqlType(SqlTypeName.DECIMAL, 14, 7);
case DOUBLE:
// the default max precision is 19, so this is actually DECIMAL(19, 15)
// but derived system can override the max precision/scale.
return createSqlType(SqlTypeName.DECIMAL, 30, 15);
{code}
The reference.md itself seems to be wrong:
{code}
| REAL, FLOAT | 4 byte floating point | 6 decimal digits precision.
| DOUBLE | 8 byte floating point | 15 decimal digits precision.
{code}
and there are many more I haven't even checked!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6073) Documentation does not specify CAST behavior
Mihai Budiu created CALCITE-6073: Summary: Documentation does not specify CAST behavior Key: CALCITE-6073 URL: https://issues.apache.org/jira/browse/CALCITE-6073 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.35.0 Reporter: Mihai Budiu I could not find documentation on how Calcite implements something like CAST(1.9 AS INT). I suspect that there are multiple implementations, some for compile-time and some for run-time. In the absence of a spec it's not clear that they agree. The SQL standard leaves this behavior at the choice of the implementation. At least RexExecutorImpl uses BigDecimal.intValue for this expression, which means that the result is 1 rather than 2, as I would have guessed. Moreover, there are many tests in SqlOperatorTests which are disabled with "brokenTestsEnabled". Most of these tests look wrong too, claiming that the above cast should fail with an error of out of range, e.g., see testCastWithRoundingToScalar. So reading the test code also does not answer this question. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6071) RexNode should carry source position information for runtime error reporting
Mihai Budiu created CALCITE-6071: Summary: RexNode should carry source position information for runtime error reporting Key: CALCITE-6071 URL: https://issues.apache.org/jira/browse/CALCITE-6071 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.35.0 Reporter: Mihai Budiu Currently runtime errors, such as division by zero, cannot report the position in the original source code where the original operator was. RexNode does not carry such information. By adding the required information in RexNode we can report much better runtime errors to users. Here is an additional comment from [~jhyde] on the dev mailing list: The RexCall to such throwable functions would have a hidden ‘pos’ parameter. The parameter should be populated at Sql-to-rel time, and if it is not present, SqlOperator.validareRexOperands should throw. Associating the position with the particular function call seems to me much more robust than associating it with the RelNode that contains the call. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6059) Optimizer does not correctly handle special floating point value -0.0E0
Mihai Budiu created CALCITE-6059:
Summary: Optimizer does not correctly handle special floating
point value -0.0E0
Key: CALCITE-6059
URL: https://issues.apache.org/jira/browse/CALCITE-6059
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
Some optimizations seem to convert -0.0E0 into 0.0E0.
In particular, the following SqlOperatorTest test fails:
f.checkScalarApprox("CAST('-0E0' AS REAL)","REAL NOT NULL", "-0E0");
This surfaced during CALCITE-6052
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6058) Parser should support special floating point values
Mihai Budiu created CALCITE-6058:
Summary: Parser should support special floating point values
Key: CALCITE-6058
URL: https://issues.apache.org/jira/browse/CALCITE-6058
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The parser should make it easier to express special floating point values, such
as NaN, Infinity, and -0.0. Right now they can be specified as CAST('Infinity'
AS DOUBLE), but perhaps a syntax such as DOUBLE '+Inf' would be nicer.
This surfaced during the discussion for [CALCITE-6052]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6052) reltosql writes FLOATING POINT literals as DECIMAL literals
Mihai Budiu created CALCITE-6052: Summary: reltosql writes FLOATING POINT literals as DECIMAL literals Key: CALCITE-6052 URL: https://issues.apache.org/jira/browse/CALCITE-6052 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.35.0 Reporter: Mihai Budiu This bug is already fixed in https://github.com/apache/calcite/pull/3411, but I plan to submit a smaller point fix for it, which doesn't require reworking the type families. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6030) DATE_PART is not handled by the RexToLixTranslator
Mihai Budiu created CALCITE-6030:
Summary: DATE_PART is not handled by the RexToLixTranslator
Key: CALCITE-6030
URL: https://issues.apache.org/jira/browse/CALCITE-6030
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following test, when added to SqlOperatorTest, causes a RuntimeException:
{code:java}
@Test void testDatePart() {
final SqlOperatorFixture f = fixture().withLibrary(SqlLibrary.POSTGRESQL)
.withParserConfig(p -> p.withParserFactory(SqlBabelParserImpl.FACTORY));
f.checkScalar("DATE_PART(second, TIME '10:10:10')",
"10", "BIGINT NOT NULL");
}
{code}
Note that this needs https://github.com/apache/calcite/pull/3445 to execute
correctly.
The stack trace is:
{code:java}
Suppressed: java.lang.RuntimeException: cannot translate call DATE_PART($t1,
$t2)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:1160)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:101)
at org.apache.calcite.rex.RexCall.accept(RexCall.java:189)
{code}
According to the documentation DATE_PART is just an alias for EXTRACT, which is
(mostly implemented), so this should work.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6029) SqlOperatorTest cannot test operators that require the Babel parser
Mihai Budiu created CALCITE-6029:
Summary: SqlOperatorTest cannot test operators that require the
Babel parser
Key: CALCITE-6029
URL: https://issues.apache.org/jira/browse/CALCITE-6029
Project: Calcite
Issue Type: Bug
Components: babel, core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
In SqlOperatorTest one can write code like this:
{code:java}
@Test void testDatePart() {
final SqlOperatorFixture f = fixture().withLibrary(SqlLibrary.POSTGRESQL)
.withParserConfig(p -> p.withParserFactory(SqlBabelParserImpl.FACTORY));
{code}
This almost works, but the SqlOperatorTest.check function makes a connection
ignores the parserFactory, so parsing will fail:
{code:java}
@Override public void check(SqlTestFactory factory, String query,
SqlTester.TypeChecker typeChecker,
SqlTester.ParameterChecker parameterChecker,
SqlTester.ResultChecker resultChecker) {
super.check(factory, query, typeChecker, parameterChecker, resultChecker);
final RelDataTypeSystem typeSystem =
factory.typeSystemTransform.apply(RelDataTypeSystem.DEFAULT);
final ConnectionFactory connectionFactory =
factory.connectionFactory
.with(CalciteConnectionProperty.TYPE_SYSTEM, uri(FIELD)); /// NO
PARSER_FACTORY HERE
{code}
I am trying to fix this by adding a PARSER_FACTORY argument to the connection,
but then I get a class loader error from AvaticaUtils.instantiatePlugin, which,
in this case, cannot find the SqlBabelParserImpl#FACTORY in the classpath.
I would appreciate some help solving this last bit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6015) AssertionError during optimization of EXTRACT expression
Mihai Budiu created CALCITE-6015:
Summary: AssertionError during optimization of EXTRACT expression
Key: CALCITE-6015
URL: https://issues.apache.org/jira/browse/CALCITE-6015
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following test added to RelOptRules test causes an AssertionError:
{code:java}
@Test void testExtractDayFromTime() {
final String sql = "select EXTRACT(DAY FROM TIME'10:00:00')";
sql(sql).withRule(CoreRules.PROJECT_REDUCE_EXPRESSIONS)
.check();
}
{code}
The bottom of the stack trace is:
{code:java}
java.lang.AssertionError: unexpected TIME
at
org.apache.calcite.adapter.enumerable.RexImpTable$ExtractImplementor.implementSafe(RexImpTable.java:3056)
at
org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.genValueStatement(RexImpTable.java:3796)
at
org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.implement(RexImpTable.java:3758)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:1184)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:101)
at org.apache.calcite.rex.RexCall.accept(RexCall.java:189)
{code}
This expression is indeed illegal. Perhaps validation should produce an error?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6014) Create a SqlOperatorFixture that parses, unparses, and then parses again before executing
Mihai Budiu created CALCITE-6014: Summary: Create a SqlOperatorFixture that parses, unparses, and then parses again before executing Key: CALCITE-6014 URL: https://issues.apache.org/jira/browse/CALCITE-6014 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.35.0 Reporter: Mihai Budiu Such a fixture will help catch bugs in the unparsing code. Several bugs were found using this technique, e.g., CALCITE-5997. This is related to CALCITE-5891, CALCITE-6000. The SqlParserFixture UnparsingTesterImpl provides a similar service, but since it does not validate the code after unparsing, it will catch fewer bugs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6006) SqlToRelConverter loses charset information
Mihai Budiu created CALCITE-6006: Summary: SqlToRelConverter loses charset information Key: CALCITE-6006 URL: https://issues.apache.org/jira/browse/CALCITE-6006 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.35.0 Reporter: Mihai Budiu This is a bug in SqlImplementor, when it calls SqlLiteral.createCharString it does not pass any information about the charset of the source string. So a string that looks like _UTF8'...' is converted to a string without the charset in the generated SQL. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6003) json_array() with no arguments does not unparse correctly
Mihai Budiu created CALCITE-6003:
Summary: json_array() with no arguments does not unparse correctly
Key: CALCITE-6003
URL: https://issues.apache.org/jira/browse/CALCITE-6003
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following expression "json_array()" is compiled by the SqlOperatorTest into
the following query:
{code:sql}
select json_array() from (values (1)) as t(p0)
{code}
Parsing and unparsing this query generates the following incorrect SQL:
{code:sql}
SELECT JSON_ARRAY(ABSENT ON NULL)
FROM (VALUES ROW(1)) AS "T" ("P0")
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6002) CONTAINS_SUBSTR does not unparse correctly
Mihai Budiu created CALCITE-6002: Summary: CONTAINS_SUBSTR does not unparse correctly Key: CALCITE-6002 URL: https://issues.apache.org/jira/browse/CALCITE-6002 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.35.0 Reporter: Mihai Budiu There are in fact two bugs related to CONTAINS_SUBSTR. The first one is that the documentation does not render in the md file, there must be something wrong with the html escape sequences. The second one is that it does not unparse into a correct form. [~tanclary] I think you added this code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6000) There should be a SqlParserFixture which parses again after unparsing
Mihai Budiu created CALCITE-6000: Summary: There should be a SqlParserFixture which parses again after unparsing Key: CALCITE-6000 URL: https://issues.apache.org/jira/browse/CALCITE-6000 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.35.0 Reporter: Mihai Budiu SqlParserTests parse and unparse queries. But the unparsed result is not validated. A new fixture should parse the final result again to validate that it's legal SQL. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5999) DECIMAL literals as sometimes unparsed looking as DOUBLE literals
Mihai Budiu created CALCITE-5999: Summary: DECIMAL literals as sometimes unparsed looking as DOUBLE literals Key: CALCITE-5999 URL: https://issues.apache.org/jira/browse/CALCITE-5999 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.35.0 Reporter: Mihai Budiu Consider a DECIMAL literal such as "0.1" When unparsed this will show up as 1E-17, which is interpreted by SQL as a double literal. The bug is in the function SqlNumericLiteral.toValue(). The function calls toString() on a BigDecimal value, but it should probably call toPlainString() instead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5998) The SAFE_OFFSET operator can cause an index out of bounds exception
Mihai Budiu created CALCITE-5998:
Summary: The SAFE_OFFSET operator can cause an index out of bounds
exception
Key: CALCITE-5998
URL: https://issues.apache.org/jira/browse/CALCITE-5998
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following query, when added as a SqlOperatorTest:
{code:sql}
select ARRAY[p3,p2,p1][SAFE_OFFSET(p0)] from (values (-1, 6, 4, 2)) as t(p0,
p1, p2, p3)
{code}
causes an exception. Here is the top of the stack trace:
{code:java}
Array index -1 is out of bounds
org.apache.calcite.runtime.CalciteException: Array index -1 is out of bounds
at
java.base@11.0.18/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base@11.0.18/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
java.base@11.0.18/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base@11.0.18/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
app//org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:507)
at
app//org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:601)
at
app//org.apache.calcite.runtime.SqlFunctions.arrayItem(SqlFunctions.java:4742)
at
app//org.apache.calcite.runtime.SqlFunctions.arrayItemOptional(SqlFunctions.java:4780)
at Baz$1$1.current(Unknown Source)
at
app//org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:687)
at
app//org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:46)
at
app//org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:219)
at
app//org.apache.calcite.sql.test.ResultCheckers.compareResultSet(ResultCheckers.java:128)
at
app//org.apache.calcite.sql.test.ResultCheckers$RefSetResultChecker.checkResult(ResultCheckers.java:336)
at
app//org.apache.calcite.test.SqlOperatorTest$TesterImpl.check(SqlOperatorTest.java:12987)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5997) OFFSET operator is incorrectly unparsed
Mihai Budiu created CALCITE-5997:
Summary: OFFSET operator is incorrectly unparsed
Key: CALCITE-5997
URL: https://issues.apache.org/jira/browse/CALCITE-5997
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following query:
{code:sql}
select ARRAY[p2,p1,p0][OFFSET(2)] from (values (6, 4, 2)) as t(p0, p1, p2)
{code}
when parsed as a SqlNode and then unparsed produces:
{code:sql}
SELECT ARRAY["P2", "P1", "P0"][2]
FROM (VALUES ROW(6, 4, 2)) AS "T" ("P0", "P1", "P2")
{code}
which no longer produces the same result (the OFFSET function call is missing).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5996) TRANSLATE operator is incorrectly unparsed
Mihai Budiu created CALCITE-5996:
Summary: TRANSLATE operator is incorrectly unparsed
Key: CALCITE-5996
URL: https://issues.apache.org/jira/browse/CALCITE-5996
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
This query
{code:sql}
select translate(col using utf8)
from (select 'a' as col
from (values(true)))
{code}
if converted to SqlNode and back produces
{code;sql}
SELECT TRANSLATE("COL", "UTF8")
FROM (SELECT 'a' AS "COL"
FROM (VALUES ROW(TRUE)))
{code}
which is no longer correct SQL.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5989) Type inference for RPAD and LPAD functions (BIGQUERY) is incorrect
Mihai Budiu created CALCITE-5989: Summary: Type inference for RPAD and LPAD functions (BIGQUERY) is incorrect Key: CALCITE-5989 URL: https://issues.apache.org/jira/browse/CALCITE-5989 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.35.0 Reporter: Mihai Budiu The type inference uses the type `ReturnTypes.ARG0_NULLABLE_VARYING` for the output. This means that the output cannot be longer than arg0. This bug surfaces when the query is optimized. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5988) SqlImplementor.toSql cannot emit VARBINARY literals
Mihai Budiu created CALCITE-5988:
Summary: SqlImplementor.toSql cannot emit VARBINARY literals
Key: CALCITE-5988
URL: https://issues.apache.org/jira/browse/CALCITE-5988
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
Given a literal with type VARBINARY the function SqlImplementor.toSql() will
crash with assertion failure error:
{code}
X'41424344':VARBINARY: BINARY
java.lang.AssertionError: X'41424344':VARBINARY: BINARY
at
org.apache.calcite.rel.rel2sql.SqlImplementor.toSql(SqlImplementor.java:1461)
at
org.apache.calcite.rel.rel2sql.SqlImplementor.toSql(SqlImplementor.java:1384)
at
org.apache.calcite.rel.rel2sql.SqlImplementor$Context.toSql(SqlImplementor.java:696)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5987) SqlImplementor loses type information for literals
Mihai Budiu created CALCITE-5987:
Summary: SqlImplementor loses type information for literals
Key: CALCITE-5987
URL: https://issues.apache.org/jira/browse/CALCITE-5987
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
When converting a SqlNode to a String query, the conversion can produce SQL
that computes different results. This happens because literals do not carry
type information in the result string. For example, this plan:
{code}
rel#7:LogicalValues.(type=RecordType(VARCHAR(3) EXPR$0),tuples=[{ 'A' }])
{code}
will generate a SQL query:
{code}
SELECT 'A'
{code}
While the type of the former result is VARCHAR(3), the latter query produces a
CHAR(1) result.
It would be nice if SqlImplementor had an option to produce a query that
preserves the output type, e.g.:
{code}
SELECT (CAST 'A' as VARCHAR(3))
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5986) The SqlTypeFamily for FP types is incorrect
Mihai Budiu created CALCITE-5986:
Summary: The SqlTypeFamily for FP types is incorrect
Key: CALCITE-5986
URL: https://issues.apache.org/jira/browse/CALCITE-5986
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
In SqlTypeFamily we have this code:
{code:java}
private static final Map JDBC_TYPE_TO_FAMILY =
...
.put(Types.FLOAT, NUMERIC)
.put(Types.REAL, NUMERIC)
.put(Types.DOUBLE, NUMERIC)
{code}
But it looks to me like the type family should be APPROXIMATE_NUMERIC.
This impacts the way RelToSqlConverter works, for instance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5981) TIMESTAMPDIFF function returns incorrect result
Mihai Budiu created CALCITE-5981:
Summary: TIMESTAMPDIFF function returns incorrect result
Key: CALCITE-5981
URL: https://issues.apache.org/jira/browse/CALCITE-5981
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following test fails, when added to SqlOperatorTest:
{code:java}
f.checkScalar("timestampdiff(month, DATE '2004-02-29', DATE '2005-02-28')",
"11", "INTEGER NOT NULL");
{code}
The result returned by the expression is 12. However, MySQL returns 11.
The semantics of this function is not described clearly in the documentation,
but according to several prior issues [1] [2] [3] the intended semantics should
be the same as in MySQL.
A corresponding MySQL test: [4]
The implementation seems to be in
StandardConvertletTable.TimestampDiffConvertlet.
[1] https://issues.apache.org/jira/browse/CALCITE-1827,
[2] https://issues.apache.org/jira/browse/CALCITE-3529,
[3] https://issues.apache.org/jira/browse/CALCITE-1124.
[4]
[https://github.com/mysql/mysql-server/blob/ea1efa9822d81044b726aab20c857d5e1b7e046a/mysql-test/r/func_time.result#L1151]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5919) Compile-time implementation of EXTRACT ignores sub-millisecond values
Mihai Budiu created CALCITE-5919:
Summary: Compile-time implementation of EXTRACT ignores
sub-millisecond values
Key: CALCITE-5919
URL: https://issues.apache.org/jira/browse/CALCITE-5919
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
When enabling the optimization rule PROJECT_REDUCE_EXPRESSIONS the compile-time
evaluation of an expression like EXTRACT(MICROSECONDS FROM TIME
'13:30:25.575401') will produce a result up to milliseconds only, irrespective
of the type system provided. (This query will evaluate to 25575000 instead of
25575401).
The bug is in RexImpTable.ExtractImplementor, here:
{code:java}
case MILLISECOND:
case MICROSECOND:
case NANOSECOND:
if (sqlTypeName == SqlTypeName.DATE) {
return Expressions.constant(0L);
}
operand = mod(operand, TimeUnit.MINUTE.multiplier.longValue(),
!isIntervalType); // << BUG
return Expressions.multiply(
operand, Expressions.constant((long) (1 /
unit.multiplier.doubleValue(;
{code}
The mod operation uses a multiplier for MINUTE that is expressed in
milliseconds, so it always truncates away values below milliseconds. The
multiplier seems to assume that the type system precision for TIME is set to 3,
which is the default value, but may be overridden.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5917) Comments in TimeString class are incorrect
Mihai Budiu created CALCITE-5917:
Summary: Comments in TimeString class are incorrect
Key: CALCITE-5917
URL: https://issues.apache.org/jira/browse/CALCITE-5917
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
Here is an example of an incorrect comment:
{code:java}
/** Sets the fraction field of a {@code TimeString} to a given number
* of milliseconds. Nukes the value set via {@link #withNanos}.
*
* For example,
* {@code new TimeString(1970, 1, 1, 2, 3, 4).withMillis(56)}
* yields {@code TIME '1970-01-01 02:03:04.056'}. */
public TimeString withMillis(int millis) {
{code}
There are several such comments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5905) Documentation for CREATE TYPE is incorrect
Mihai Budiu created CALCITE-5905:
Summary: Documentation for CREATE TYPE is incorrect
Key: CALCITE-5905
URL: https://issues.apache.org/jira/browse/CALCITE-5905
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
I have tried to run the example CREATE TYPE statements from this page
https://calcite.apache.org/docs/reference.html#declaring-objects-for-user-defined-types
through the compiler:
{code:sql}
CREATE TYPE address_typ AS OBJECT (
street VARCHAR2(30),
cityVARCHAR2(20),
state CHAR(2),
postal_code VARCHAR2(6));
{code}
Calcite complains in two places: OBJECT and VARCHAR2.
The following compiles fine:
{code:sql}
CREATE TYPE address_typ AS (
street VARCHAR(30),
cityVARCHAR(20),
state CHAR(2),
postal_code VARCHAR(6));
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5901) Compiler should not accept DECIMAL types with precision < scale
Mihai Budiu created CALCITE-5901:
Summary: Compiler should not accept DECIMAL types with precision <
scale
Key: CALCITE-5901
URL: https://issues.apache.org/jira/browse/CALCITE-5901
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following test, when added to to SqlOperatorTest, passes:
{code:java}
f.checkScalar("cast(0.012 as DECIMAL(2, 5))", new BigDecimal("0.012"),
"DECIMAL(2, 5) NOT NULL");
{code}
However, according to the SQL standard "The of an
shall not be greater than the of the ." In this
case the precision is 2 and the scale is 5.
So I would expect this test to fail. The validator should reject such types. I
think that this change would affect quite a few tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5891) Create a test fixture that would apply PROJECT_REDUCE_EXPRESSIONS to all tests in SqlOperatorTest
Mihai Budiu created CALCITE-5891:
Summary: Create a test fixture that would apply
PROJECT_REDUCE_EXPRESSIONS to all tests in SqlOperatorTest
Key: CALCITE-5891
URL: https://issues.apache.org/jira/browse/CALCITE-5891
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
SqlOperatorTest has many tests, including end-to-end tests.
However, none of these tests exercise the PROJECT_REDUCE_EXPRESSION rules,
which often produce different results than these tests for constant expressions.
Ideally we should be able to subclass SqlOperatorTest and use a fixture that
also applies this optimization prior to evaluation.
I have marked this as a {*}major priority{*}, because I suspect it would catch
many bugs with minimal effort. (I have found at least 10 so far.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5884) Nullability of of ARRAY_TO_STRING result does not depend on third argument nullability
Mihai Budiu created CALCITE-5884:
Summary: Nullability of of ARRAY_TO_STRING result does not depend
on third argument nullability
Key: CALCITE-5884
URL: https://issues.apache.org/jira/browse/CALCITE-5884
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
This is the current definition of the function ARRAY_TO_STRING in
SqlLibraryOperators:
{code:java}
/** The "ARRAY_TO_STRING(array, delimiter [, nullText ])" function. */
@LibraryOperator(libraries = {BIG_QUERY})
public static final SqlFunction ARRAY_TO_STRING =
SqlBasicFunction.create(SqlKind.ARRAY_TO_STRING,
ReturnTypes.VARCHAR_NULLABLE,
OperandTypes.STRING_ARRAY_CHARACTER_OPTIONAL_CHARACTER);
{code}
So the result is nullable if any of the arguments is nullable. However, the
nullability of the last argument does not influence the result nullabillity: a
NULL value for the third optional argument will not cause a NULL value to be
output.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5882) Compile-time evaluation of SPLIT function returns incorrect result
Mihai Budiu created CALCITE-5882:
Summary: Compile-time evaluation of SPLIT function returns
incorrect result
Key: CALCITE-5882
URL: https://issues.apache.org/jira/browse/CALCITE-5882
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The compile-time evaluation of SPLIT functions produces wrong results. Here is
an example test for RelOptRulesTest:
{code:java}
@Test public void testSplit2() {
final String query = "select split('1|2|3', '|')";
sql(query)
.withFactory(
t -> t.withOperatorTable(opTab ->
SqlLibraryOperatorTableFactory.INSTANCE.getOperatorTable(
SqlLibrary.BIG_QUERY))) // needed for SPLIT function
.withRule(CoreRules.PROJECT_REDUCE_EXPRESSIONS)
.check();
}
{code}
The result is expected to be an ARRAY containing strings '1', '2', '3', but the
result is:
{code}
LogicalProject(EXPR$0=[ARRAY('1', '2', '3')])
LogicalValues(tuples=[[{ 0 }]])
{code}
This probably happens because the type inference rule for the SPLIT operator is
(SqlLibraryOperators.java):
{code:java}
@LibraryOperator(libraries = {BIG_QUERY})
public static final SqlFunction SPLIT =
SqlBasicFunction.create("SPLIT",
ReturnTypes.ARG0
.andThen(SqlLibraryOperators::deriveTypeSplit)
.andThen(SqlTypeTransforms.TO_ARRAY),
...
{code}
Thus, when ARG0 is CHAR(4) the return type is also inferred to be CHAR(4).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5879) Calcite crashes with AssertionFailure during evaluation of constant SPLIT expression
Mihai Budiu created CALCITE-5879:
Summary: Calcite crashes with AssertionFailure during evaluation
of constant SPLIT expression
Key: CALCITE-5879
URL: https://issues.apache.org/jira/browse/CALCITE-5879
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
The following test, when inserted in RelOptRulesTest, crashes at compilation
time with an AssertionFailure:
{code:java}
@Test public void testSplit() {
final String query = "select split('1|2|3', NULL)";
sql(query)
.withFactory(
t -> t.withOperatorTable(opTab ->
SqlLibraryOperatorTableFactory.INSTANCE.getOperatorTable(
SqlLibrary.BIG_QUERY))) // needed for SPLIT function
.withRule(CoreRules.PROJECT_REDUCE_EXPRESSIONS)
.check();
}
{code}
The error message and stack trace look like this:
{code:java}
java.lang.AssertionError: Cannot add expression of different type to set:
set type is RecordType(CHAR(5) NOT NULL ARRAY NOT NULL EXPR$0) NOT NULL
expression type is RecordType(CHAR(5) NOT NULL ARRAY EXPR$0) NOT NULL
set is rel#4:LogicalProject.(input=HepRelVertex#3,exprs=[SPLIT('1|2|3',
null:NULL)])
expression is LogicalProject(EXPR$0=[null:CHAR(5) NOT NULL ARRAY])
LogicalValues(tuples=[[{ 0 }]])
Type mismatch:
rowtype of original rel: RecordType(CHAR(5) NOT NULL ARRAY NOT NULL EXPR$0) NOT
NULL
rowtype of new rel: RecordType(CHAR(5) NOT NULL ARRAY EXPR$0) NOT NULL
Difference:
EXPR$0: CHAR(5) NOT NULL ARRAY NOT NULL -> CHAR(5) NOT NULL ARRAY
at
org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:394)
at
org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:60)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5877) Calcite crashes with AssertionError when compiling MOD with arguments with large scales
Mihai Budiu created CALCITE-5877:
Summary: Calcite crashes with AssertionError when compiling MOD
with arguments with large scales
Key: CALCITE-5877
URL: https://issues.apache.org/jira/browse/CALCITE-5877
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Mihai Budiu
Adding the following test to RelToSqlConverterTest produces an AssertionFailure:
{code:java}
@Test void testNumericScaleMod() {
final String sql = "SELECT MOD(CAST(2 AS DECIMAL(39, 20)), 2)";
final String expected = "";
sql(sql).withPostgresqlModifiedDecimalTypeSystem()
.ok(expected);
}
{code}
The exception is here:
{code:java}
java.lang.AssertionError
at
org.apache.calcite.rel.type.RelDataTypeSystem.deriveDecimalModType(RelDataTypeSystem.java:383)
at
org.apache.calcite.sql.type.ReturnTypes.lambda$static$11(ReturnTypes.java:849)
at
org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:58)
at
org.apache.calcite.sql.type.SqlReturnTypeInferenceChain.inferReturnType(SqlReturnTypeInferenceChain.java:55)
{code}
In this line:
{code:java}
int scale = Math.max(s1, s2);
assert scale <= getMaxNumericScale();
{code}
I suspect that doing a Math.max with the getMaxNumericScale (instead of an
Assert) is sufficient to fix this particular problem. The question is whether
the compiler should really enforce the invariant that this assertion checks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5861) Optimization rules do not constant-fold expressions in window bounds
Mihai Budiu created CALCITE-5861:
Summary: Optimization rules do not constant-fold expressions in
window bounds
Key: CALCITE-5861
URL: https://issues.apache.org/jira/browse/CALCITE-5861
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.34.0
Reporter: Mihai Budiu
This bug is a bit more complicated to describe, and there is a possibility that
I am doing something wrong.
Consider the following test that can be added to RelOptRulesTest:
{code:java}
@Test void testExpressionPreceding() {
HepProgramBuilder preBuilder = new HepProgramBuilder();
preBuilder.addRuleInstance(CoreRules.WINDOW_REDUCE_EXPRESSIONS);
preBuilder.addRuleInstance(CoreRules.PROJECT_TO_LOGICAL_PROJECT_AND_WINDOW);
HepProgramBuilder builder = new HepProgramBuilder();
builder.addRuleInstance(CoreRules.PROJECT_REDUCE_EXPRESSIONS);
HepPlanner hepPlanner = new HepPlanner(builder.build());
final String sql =
"select COUNT(*) over (\n"
+ "ORDER BY empno\n"
+ "ROWS BETWEEN 5 + 5 PRECEDING AND 1 PRECEDING) AS w_avg\n"
+ " from emp\n";
sql(sql)
.withPre(preBuilder.build())
.withPlanner(hepPlanner)
.check();
}
{code}
The plan before looks like this:
{code:java}
LogicalProject($0=[$2])
LogicalWindow(window#0=[window(order by [0] rows between $1 PRECEDING and $2
PRECEDING aggs [COUNT()])])
LogicalProject(EMPNO=[$0], $1=[+(5, 5)])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{code}
The problem is that the expression "5+5", which appears in the window bounds,
has not been reduced to a constant by the rule WINDOW_REDUCE_EXPRESSIONS.
Moreover, the next optimization rule PROJECT_TO_LOGICAL_PROJECT_AND_WINDOW has
pushed this expression into the LogicalProject. So it appears locally that the
LogicalWindow no longer has a constant bound, which is required by the SQL
language spec (it is constant, but that is no longer apparent in the query).
(At least our code generator is upset by this state of affairs.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5859) Calcite crashes at compilation time when evaluating LEFT(NULL, n) expression
Mihai Budiu created CALCITE-5859:
Summary: Calcite crashes at compilation time when evaluating
LEFT(NULL, n) expression
Key: CALCITE-5859
URL: https://issues.apache.org/jira/browse/CALCITE-5859
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.34.0
Reporter: Mihai Budiu
{code:sql}
SELECT LEFT(NULL, 100)
{code}
causes this exception:
{code}
Caused by: java.lang.RuntimeException: while resolving method 'valueOf[class
java.lang.String]' in class class java.lang.Void
at org.apache.calcite.linq4j.tree.Types.lookupMethod(Types.java:318)
at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:449)
at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:461)
at
org.apache.calcite.adapter.enumerable.EnumUtils.convert(EnumUtils.java:396)
at
org.apache.calcite.adapter.enumerable.EnumUtils.convert(EnumUtils.java:339)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5843) Legal SQL expression causes a parse error
Mihai Budiu created CALCITE-5843:
Summary: Legal SQL expression causes a parse error
Key: CALCITE-5843
URL: https://issues.apache.org/jira/browse/CALCITE-5843
Project: Calcite
Issue Type: Bug
Components: linq4j
Affects Versions: 1.34.0
Reporter: Mihai Budiu
This program fails at compilation time:
{code:sql}
SELECT CAST(CAST('32767.4' AS FLOAT) AS SMALLINT)
{code}
This happens in linq4j in Expressions.constant:
{code:java}
if ((clazz != Float.class && clazz != Double.class || !(value instanceof
BigDecimal)) && !clazz.isInstance(value)) {
String stringValue = String.valueOf(value);
if (type == BigDecimal.class) {
value = new BigDecimal(stringValue);
}
if (type == BigInteger.class) {
value = new BigInteger(stringValue);
}
if (primitive != null) {
value = primitive.parse(stringValue); // << error happens
here
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5813) Type inference for REPEAT sql function is incorrect
Mihai Budiu created CALCITE-5813:
Summary: Type inference for REPEAT sql function is incorrect
Key: CALCITE-5813
URL: https://issues.apache.org/jira/browse/CALCITE-5813
Project: Calcite
Issue Type: Bug
Reporter: Mihai Budiu
The result of the following SQL query (enabling the proper dialects for the
REPEAT function):
{code:sql}
SELECT REPEAT('abc', 2)
{code}
is incorrectly computed by Calcite as 'abc' (no repetitions) if the constant
folding optimization PROJECT_REDUCE_EXPRESSIONS is enabled.
(I am not sure exactly how to modify the operator tables of the RelOptFixture,
so I had to jump through some hoops to create a simple reproduction.)
The plans before and after are as following:
{code:java}
LogicalProject(EXPR$0=[REPEAT('abc', 2)])
LogicalValues(tuples=[[{ 0 }]])
---
LogicalProject(EXPR$0=['abc':VARCHAR(3)])
LogicalValues(tuples=[[{ 0 }]])
{code}
The root cause is the following:
In the definition of the REPEAT SqlFunction:
{code:java}
@LibraryOperator(libraries = {BIG_QUERY, MYSQL, POSTGRESQL})
public static final SqlFunction REPEAT =
SqlBasicFunction.create("REPEAT",
ReturnTypes.ARG0_NULLABLE_VARYING, /// <<< WRONG
OperandTypes.STRING_INTEGER,
SqlFunctionCategory.STRING);
{code}
the output type is the same as the first argument type. If the first argument
type is VARCHAR(N), the output type is also VARCHAR(N). This causes the
optimizer to first correctly compute the repeated string and then truncate
result to the original length.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5811) Error messages produced for constant out-of-bounds arguments are confusing
Mihai Budiu created CALCITE-5811:
Summary: Error messages produced for constant out-of-bounds
arguments are confusing
Key: CALCITE-5811
URL: https://issues.apache.org/jira/browse/CALCITE-5811
Project: Calcite
Issue Type: Bug
Affects Versions: 1.34.0
Reporter: Mihai Budiu
Supplying arguments that are out-of-bounds for functions can produce
uninformative messages. Consider this test case that can be added to
SqlOperatorTest.checkSubstringFunction():
{code:java}
f.checkString("substring('abc' from 2 for 2147483650)",
"bc", "VARCHAR(3) NOT NULL");
{code}
The signature of the substring function requires int arguments. The constant
2147483650 is out of bounds for an integer. This causes the test to fail with
the following exception:
{code:java}
Error while executing SQL "values (substring('abc' from 2 for 2147483650))":
Unable to implement EnumerableCalc(expr#0=[{inputs}], expr#1=['abc'],
expr#2=[2], expr#3=[2147483650:BIGINT], expr#4=[SUBSTRING($t1, $t2, $t3)],
EXPR$0=[$t4]): rowcount = 1.0, cumulative cost = {2.0 rows, 7.0 cpu, 0.0 io},
id = 153424
EnumerableValues(tuples=[[{ 0 }]]): rowcount = 1.0, cumulative cost = {1.0
rows, 1.0 cpu, 0.0 io}, id = 153408
{code}
I suspect this happens because of a combination of features:
- type inference does not reject the call to substring, although the inferred
type of the argument is probably bigint
- an attempt to evaluate the constant function fails because of some overflow
- the detailed exception about the overflow is lost when handling the error
I suspect this is a deeper problem which may affect all functions, and it is
really about type inference and implicit cast insertion.
I can imagine two possible implementation strategies:
- Postgres rejects such a call because the substring function cannot take a
bigint argument
- The compiler could add an implicit cast from bigint to int, which should at
least give a warning because the value is too large to fit in an integer
Both these solutions would avoid a crash in the expression evaluation. I
personally do not yet understand well enough the type inference mechanisms in
Calcite to propose a solution.
To make matters worse, some SQL dialects have substring functions with bigint
arguments. In this case there should be two different substring functions, with
different signatures.
This issue surfaced during a discussion for
[https://github.com/apache/calcite/pull/3286,] and is related to
https://issues.apache.org/jira/browse/CALCITE-5810.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5810) SUBSTRING compile-time evaluation gives wrong result for large lengths
Mihai Budiu created CALCITE-5810:
Summary: SUBSTRING compile-time evaluation gives wrong result for
large lengths
Key: CALCITE-5810
URL: https://issues.apache.org/jira/browse/CALCITE-5810
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.34.0
Reporter: Mihai Budiu
This program
{code:sql}
SELECT SUBSTRING('string' FROM 2 FOR 2147483646)
{code}
returns "" instead of "tring".
This happens because of an overflow in the SqlFunctions substring function:
{code:java}
public static String substring(String c, int s, int l) {
int lc = c.length();
int e = s + l; // OVERFLOW here
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5795) Type inference for VALUES with numeric values infers incorrect scale
Mihai Budiu created CALCITE-5795: Summary: Type inference for VALUES with numeric values infers incorrect scale Key: CALCITE-5795 URL: https://issues.apache.org/jira/browse/CALCITE-5795 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.34.0 Reporter: Mihai Budiu Consider this query (using the Babel parser for the :: cast notation from Postgres) WITH v(x) AS (VALUES(0::numeric),(4.2)) SELECT x FROM v as v1(x) Calcite simplifies this to 0, 4, simplified at compilation time. However, Postgres returns 0, 4.2, as expected. It seems that this happens because the type inference for VALUES infers a scale of 0. Note that the following variants of the query give correct results: WITH v(x) AS (VALUES(0::numeric),(4.2::numeric)) SELECT x FROM v as v1(x) WITH v(x) AS (VALUES(0),(4.2)) SELECT x FROM v as v1(x) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5761) DATE_TRUNC compile-time evaluation produces wrong results
Mihai Budiu created CALCITE-5761:
Summary: DATE_TRUNC compile-time evaluation produces wrong results
Key: CALCITE-5761
URL: https://issues.apache.org/jira/browse/CALCITE-5761
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.34.0
Reporter: Mihai Budiu
Adding the following 3 tests to SqlOperatorTest.testDateTrunc produces
assertion failures.
f.checkScalar("date_trunc(date '2015-02-19', decade)",
"2010-01-01", "DATE NOT NULL");
f.checkScalar("date_trunc(date '2015-02-19', century)",
"2001-01-01", "DATE NOT NULL");
f.checkScalar("date_trunc(date '2015-02-19', millennium)",
"1001-01-01", "DATE NOT NULL");
Results produced:
Query: values (date_trunc(date '2015-02-19', decade))
Expected: is "2010-01-01"
but: was "2015-01-05"
Query: values (date_trunc(date '2015-02-19', century))
Expected: is "2001-01-01"
but: was "2012-09-17"
Query: values (date_trunc(date '2015-02-19', millennium))
Expected: is "1001-01-01"
but: was "2002-11-09"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5760) SqlOperatorTest::testDateTrunc does not validate results
Mihai Budiu created CALCITE-5760:
Summary: SqlOperatorTest::testDateTrunc does not validate results
Key: CALCITE-5760
URL: https://issues.apache.org/jira/browse/CALCITE-5760
Project: Calcite
Issue Type: Bug
Components: tests
Affects Versions: 1.34.0
Reporter: Mihai Budiu
I was trying to reproduce a bug where DATE_TRUNC returns an incorrect result
when truncating a constant date to a MILLENNIUM.
I found test cases for DATE_TRUNC in SqlOperatorTest; these test cases do
specify the expected result, e.g.:
f.checkScalar("date_trunc(date '2015-02-19', isoyear)",
"2014-12-29", "DATE NOT NULL");
However, the test fixture uses AbstractSqlTester::check, which ignores the
result!
As a proof, one can replace the results with arbitrary strings and the tests
will still pass.
I presume that this is not the intent of the test writers. How could this test
be fixed to also check the results?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5651) Type system decimal scale in inferred types may exceed allowed scale
Mihai Budiu created CALCITE-5651:
Summary: Type system decimal scale in inferred types may exceed
allowed scale
Key: CALCITE-5651
URL: https://issues.apache.org/jira/browse/CALCITE-5651
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.34.0
Reporter: Mihai Budiu
If one uses a type system with a scale which is < precision/2 the calcite
compiler may crash with an assertion failure. This can happen in the
RelDataTypeSystem.deriveDecimalPlusType method, where this assertion fails:
assert scale <= getMaxNumericScale();
The root cause is the function SqlTypeUtil.getMaxPrecisionScaleDecimal, which
does the following:
int scale = maxPrecision / 2;
This was introduced in Calcite 1.27.
I think the a better version should be
int scale = Math.min(maxPrecision/2,
factory.getTypeSystem().getMaxNumericScale());
To reproduce this bug I had to do a bit of juggling. First I had to fix the bug
reported in Calcite-5650. Then I added the following two functions in
RelToSqlConverterTest.java:
@Test void testNumericScale() {
String q = "WITH v(x) AS (VALUES('4.2')) " +
" SELECT x1 + x2 FROM v AS v1(x1), v AS V2(x2)";
sql(q)
.withPostgresqlModifiedDecimalTypeSystem()
.ok("");
}
and in the static class Sql I added this method:
Sql withPostgresqlModifiedDecimalTypeSystem() {
final PostgresqlSqlDialect postgresqlSqlDialect =
new PostgresqlSqlDialect(PostgresqlSqlDialect.DEFAULT_CONTEXT
.withDataTypeSystem(new RelDataTypeSystemImpl() {
@Override
public int getMaxNumericScale() {
return 10;
}
@Override
public int getMaxNumericPrecision() {
return 39;
}
}));
return dialect(postgresqlSqlDialect);
}
This is the exception stack trace:
java.lang.AssertionError
at
org.apache.calcite.rel.type.RelDataTypeSystem.deriveDecimalPlusType(RelDataTypeSystem.java:167)
at
org.apache.calcite.sql.type.ReturnTypes.lambda$static$8(ReturnTypes.java:653)
at
org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:58)
at
org.apache.calcite.sql.type.SqlReturnTypeInferenceChain.inferReturnType(SqlReturnTypeInferenceChain.java:55)
at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:537)
at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:504)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:605)
at
org.apache.calcite.sql.SqlBinaryOperator.deriveType(SqlBinaryOperator.java:178)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6521)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6508)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:161)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1897)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1882)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:489)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4608)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3780)
at
org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:61)
at
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:88)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1135)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1106)
at
org.apache.calcite.sql.validate.WithNamespace.validateImpl(WithNamespace.java:59)
at
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:88)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1135)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateWith(SqlValidatorImpl.java:4240)
at org.apache.calcite.sql.SqlWith.validate(SqlWith.java:74)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1081)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:787)
at org.apache.calcite.prepare.PlannerImpl.validate(PlannerImpl.java:226)
at
org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.exec(RelToSqlConverterTest.java:7076)
at
org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.ok(RelToSqlConverterTest.java:7043)
at
[jira] [Created] (CALCITE-5650) Sql.dialect method in RelToSqlConverterTests ignores dialect type system
Mihai Budiu created CALCITE-5650:
Summary: Sql.dialect method in RelToSqlConverterTests ignores
dialect type system
Key: CALCITE-5650
URL: https://issues.apache.org/jira/browse/CALCITE-5650
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.34.0
Reporter: Mihai Budiu
The problem is in the file SqlToRelConverterTest.java, in the static class Sql,
in the following method:
Sql dialect(SqlDialect dialect) {
return new Sql(schemaSpec, sql, dialect, parserConfig, librarySet, config,
relFn, transforms, typeSystem);
}
The problem is that the dialect may have a different type system, which is
ignored in this function. I think the function should read:
Sql dialect(SqlDialect dialect) {
return new Sql(schemaSpec, sql, dialect, parserConfig, librarySet, config,
relFn, transforms, dialect.getTypeSystem());
}
Several tests in this file do change the dialect type system.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5638) Assertion Failure during planning correlated query with orderby
Mihai Budiu created CALCITE-5638:
Summary: Assertion Failure during planning correlated query with
orderby
Key: CALCITE-5638
URL: https://issues.apache.org/jira/browse/CALCITE-5638
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.34.0
Reporter: Mihai Budiu
Here is a test which fails if pasted in jdbcTest.java:
@Test void testCrash() throws Exception {
String hsqldbMemUrl = "jdbc:hsqldb:mem:.";
Connection baseConnection = DriverManager.getConnection(hsqldbMemUrl);
Statement baseStmt = baseConnection.createStatement();
baseStmt.execute("CREATE TABLE t1(a INTEGER, b INTEGER, c INTEGER, d
INTEGER, e INTEGER)");
baseStmt.close();
baseConnection.commit();
Properties info = new Properties();
info.put("model",
"inline:"
+ "{\n"
+ " version: '1.0',\n"
+ " defaultSchema: 'BASEJDBC',\n"
+ " schemas: [\n"
+ " {\n"
+ " type: 'jdbc',\n"
+ " name: 'BASEJDBC',\n"
+ " jdbcDriver: '" + jdbcDriver.class.getName() + "',\n"
+ " jdbcUrl: '" + hsqldbMemUrl + "',\n"
+ " jdbcCatalog: null,\n"
+ " jdbcSchema: null\n"
+ " }\n"
+ " ]\n"
+ "}");
Connection calciteConnection =
DriverManager.getConnection("jdbc:calcite:", info);
String statement = "SELECT b, d, (SELECT count(*) FROM t1 AS x WHERE
x.c>t1.c AND x.dc ORDER BY 1,2,3";
ResultSet rs = calciteConnection.prepareStatement(statement).executeQuery();
rs.close();
calciteConnection.close();
}
The stack trace is:
Required columns \{2, 3} not subset of left columns \{0, 1, 2}
java.lang.AssertionError: Required columns \{2, 3} not subset of left columns
\{0, 1, 2}
at org.apache.calcite.util.Litmus.lambda$static$0(Litmus.java:31)
at org.apache.calcite.util.Litmus.check(Litmus.java:76)
at org.apache.calcite.rel.core.Correlate.isValid(Correlate.java:145)
at org.apache.calcite.rel.core.Correlate.(Correlate.java:109)
at
org.apache.calcite.rel.logical.LogicalCorrelate.(LogicalCorrelate.java:72)
at
org.apache.calcite.rel.logical.LogicalCorrelate.create(LogicalCorrelate.java:115)
at
org.apache.calcite.rel.core.RelFactories$CorrelateFactoryImpl.createCorrelate(RelFactories.java:440)
at org.apache.calcite.tools.RelBuilder.join(RelBuilder.java:2865)
at
org.apache.calcite.rel.rules.SubQueryRemoveRule.rewriteScalarQuery(SubQueryRemoveRule.java:136)
at
org.apache.calcite.rel.rules.SubQueryRemoveRule.apply(SubQueryRemoveRule.java:94)
at
org.apache.calcite.rel.rules.SubQueryRemoveRule.matchProject(SubQueryRemoveRule.java:828)
at
org.apache.calcite.rel.rules.SubQueryRemoveRule.access$200(SubQueryRemoveRule.java:75)
at
org.apache.calcite.rel.rules.SubQueryRemoveRule$Config.lambda$static$0(SubQueryRemoveRule.java:906)
at
org.apache.calcite.rel.rules.SubQueryRemoveRule.onMatch(SubQueryRemoveRule.java:86)
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:556)
at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:420)
at
org.apache.calcite.plan.hep.HepPlanner.executeRuleCollection(HepPlanner.java:286)
at
org.apache.calcite.plan.hep.HepInstruction$RuleCollection$State.execute(HepInstruction.java:105)
at
org.apache.calcite.plan.hep.HepPlanner.lambda$executeProgram$0(HepPlanner.java:211)
at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422)
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:210)
at org.apache.calcite.plan.hep.HepProgram$State.execute(HepProgram.java:118)
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:205)
at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:191)
at org.apache.calcite.tools.Programs.lambda$of$0(Programs.java:177)
at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:337)
at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:177)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:312)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:665)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:519)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:487)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:236)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:216)
at
[jira] [Created] (CALCITE-5615) Run SQLLogicTests using Calcite
Mihai Budiu created CALCITE-5615: Summary: Run SQLLogicTests using Calcite Key: CALCITE-5615 URL: https://issues.apache.org/jira/browse/CALCITE-5615 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.34.0 Reporter: Mihai Budiu Sqllogictest is a program designed to verify that an SQL database engine computes correct results by comparing the results to identical queries from other SQL database engines. https://www.sqlite.org/sqllogictest/doc/trunk/about.wiki The nice thing about SLT is that it contains more than 7 million tests. The tests only cover the core of SQL, ideally the portable part across all engines. They only test integers, doubles, and strings. So they could probably be part of the Calcite slow tests. The tests should be structured so that any query execution engine can be used. I plan to contribute such an implementation if people think it is useful, but I haven't yet worked out all the details. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5538) Calcite rejects timestamp literals that end with 0 after the period
Mihai Budiu created CALCITE-5538: Summary: Calcite rejects timestamp literals that end with 0 after the period Key: CALCITE-5538 URL: https://issues.apache.org/jira/browse/CALCITE-5538 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.33.0 Reporter: Mihai Budiu The TimestampString constructor validates a string argument with the following regular expression: "[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]" + " " + "[0-9][0-9]:[0-9][0-9]:[0-9][0-9](\\.[0-9]*[1-9])?" Unfortunately this rejects perfectly legal timestamp strings such as "2023-02-21 10:10:10.000". The fix is trivial, if we agree that this is a bug. Is there a deeper reason for this validation? There seem to be no unit tests for this TimestampString constructor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5448) ReduceExpressionsRule does not always constant fold expressions
Mihai Budiu created CALCITE-5448:
Summary: ReduceExpressionsRule does not always constant fold
expressions
Key: CALCITE-5448
URL: https://issues.apache.org/jira/browse/CALCITE-5448
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.32.0
Reporter: Mihai Budiu
I have manually built a HepPlanner to optimize the SQL queries, and I
discovered that the rule ReduceExpressionsRule does not really do anything in
my setup.
I am looking at method ReduceExpressionsRule.reduceExpressionsInternal.
There is this piece of code:
RexExecutor executor = rel.getCluster().getPlanner().getExecutor();
if (executor == null) {
// Cannot reduce expressions: caller has not set an executor in their
// environment. Caller should execute something like the following before
// invoking the planner:
//
// final RexExecutorImpl executor =
// new RexExecutorImpl(Schemas.createDataContext(null));
// rootRel.getCluster().getPlanner().setExecutor(executor);
return changed;
}
However, the caller of this function, the method reduceExpressions, has
carefully inserted an executor in the RexSimplify class in case the cluster has
no executor. Shouldn't that executor be used instead of trying the missing one?
(It is currently private in the RexSimplify class.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5343) Type of division operator
Mihai Budiu created CALCITE-5343: Summary: Type of division operator Key: CALCITE-5343 URL: https://issues.apache.org/jira/browse/CALCITE-5343 Project: Calcite Issue Type: Bug Components: core Reporter: Mihai Budiu The Calcite type checker assigns a type of INTEGER (not nullable) to the DIVIDE RexCall in the following statement: `SELECT 1/0`. Some databases, such as MySQL, evaluate this expression to NULL. This result is obtained even when using SqlConformanceEnum.MYSQL_5 for the parser and the validator. My question is whether there is some other way to influence how type inference is performed for division. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5289) Assertion failure in MultiJoinOptimizeBushyRule
Mihai Budiu created CALCITE-5289:
Summary: Assertion failure in MultiJoinOptimizeBushyRule
Key: CALCITE-5289
URL: https://issues.apache.org/jira/browse/CALCITE-5289
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.32.0
Reporter: Mihai Budiu
The reproduction is easy: just modify the following test case from
PlannerTest.java:
{{{}--- a/core/src/test/java/org/apache/calcite/tools/PlannerTest.java
{}}}{{{}+++ b/core/src/test/java/org/apache/calcite/tools/PlannerTest.java
{}}}{{{}@@ -1005,7 +1005,7 @@ private void checkJoinNWay(int n) throws
Exception {{}}}
{{private void checkHeuristic(String sql, String expected) throws Exception {
}}{{{} Planner planner = getPlanner(null,
{}}}{{{}- Programs.heuristicJoinOrder(Programs.RULE_SET, false, 0));
{}}}{{{}+ Programs.heuristicJoinOrder(Programs.RULE_SET, true, 0));
{}}}{{ SqlNode parse = planner.parse(sql);
}}{{ SqlNode validate = planner.validate(parse);
}}{{ RelNode convert = planner.rel(validate).rel;}}
Then the test fails with the exception shown below. This happens with the
latest version of calcite, the main branch.
It looks like the rule does not account for the fact that outer joins can
produce results with a different nullability than the input relations.
The exception can be triggered even for very simple outer join queries, e.g.:
SELECT T1.COL3 FROM T AS T1 LEFT JOIN T AS T2 ON T1.COL1 = T2.COL5
The only workaround I found is to make sure this rule is never applied when a
query contains an outer join.
Here is the Java stack trace:
{{java.lang.RuntimeException: Error while applying rule
MultiJoinOptimizeBushyRule, args
[rel#44:MultiJoin.NONE.[](input#0=RelSubset#42,input#1=RelSubset#43,joinFilter=true,isFullOuterJoin=false,joinTypes=[RIGHT,
INNER],outerJoinConditions=[=($0, $10), NULL],projFields=[ALL, ALL])]}}
{{ at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:250)}}
{{ at
org.apache.calcite.plan.volcano.IterativeRuleDriver.drive(IterativeRuleDriver.java:59)}}
{{ at
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:523)}}
{{ at
org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:318)}}
{{ at
org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:337)}}
{{ at
org.apache.calcite.tools.Programs.lambda$heuristicJoinOrder$1(Programs.java:223)}}
{{ at
org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:373)}}
{{ at
org.apache.calcite.tools.PlannerTest.checkHeuristic(PlannerTest.java:1014)}}
{{ at
org.apache.calcite.tools.PlannerTest.testHeuristicRightJoin(PlannerTest.java:1003)}}
{{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)}}
{{ at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
{{ at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
{{ at java.lang.reflect.Method.invoke(Method.java:498)}}
{{ at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)}}
{{ at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)}}
{{ at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)}}
{{ at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)}}
{{ at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)}}
{{ at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)}}
{{ at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)}}
{{ at
org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)}}
{{ at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)}}
{{ at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)}}
{{ at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)}}
{{ at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)}}
{{ at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)}}
{{ at
