[jira] [Created] (CALCITE-4819) SemiJoin operator is not skipped in materialized view-based rewriting algorithm
Jesus Camacho Rodriguez created CALCITE-4819: Summary: SemiJoin operator is not skipped in materialized view-based rewriting algorithm Key: CALCITE-4819 URL: https://issues.apache.org/jira/browse/CALCITE-4819 Project: Calcite Issue Type: Bug Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez This had been solved in CALCITE-2277 but apparently it was broken in CALCITE-2696 (MaterializedViewRelOptRulesTest.testJoinMaterialization11 result was modified assuming the change was correct). The rewriting algorithm does not support semijoin so this can lead to incorrect results. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4716) ClassCastException converting SARG in RelNode to SQL
Jesus Camacho Rodriguez created CALCITE-4716:
Summary: ClassCastException converting SARG in RelNode to SQL
Key: CALCITE-4716
URL: https://issues.apache.org/jira/browse/CALCITE-4716
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
The stacktrace is the following:
{noformat}
class org.apache.calcite.rex.RexLocalRef cannot be cast to class
org.apache.calcite.rex.RexLiteral (org.apache.calcite.rex.RexLocalRef and
org.apache.calcite.rex.RexLiteral are in unnamed module of loader 'app')
java.lang.ClassCastException: class org.apache.calcite.rex.RexLocalRef cannot
be cast to class org.apache.calcite.rex.RexLiteral
(org.apache.calcite.rex.RexLocalRef and org.apache.calcite.rex.RexLiteral are
in unnamed module of loader 'app')
at
org.apache.calcite.rel.rel2sql.SqlImplementor$Context.toSql(SqlImplementor.java:695)
at
org.apache.calcite.rel.rel2sql.SqlImplementor$Context.toSql(SqlImplementor.java:597)
...
{noformat}
The relevant expressions in the Calc operator are the following:
{code}
...expr#5=[Sarg[(10..11]]], expr#6=[SEARCH($t0, $t5)]...
{code}
The current code in {{SqlImplementor}} considers the second argument to SEARCH
is always a RexLiteral:
{code}
...
case SEARCH:
final RexCall search = (RexCall) rex;
literal = (RexLiteral) search.operands.get(1);
final Sarg sarg = castNonNull(literal.getValueAs(Sarg.class));
//noinspection unchecked
return toSql(program, search.operands.get(0), literal.getType(), sarg);
...
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4499) FilterJoinRule misses opportunity to push filter to semijoin input
Jesus Camacho Rodriguez created CALCITE-4499:
Summary: FilterJoinRule misses opportunity to push filter to
semijoin input
Key: CALCITE-4499
URL: https://issues.apache.org/jira/browse/CALCITE-4499
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Consider the following plan:
{code}
LogicalProject(DNAME=[$1])
LogicalJoin(condition=[AND(=($0, $10), =($8, 100))], joinType=[semi])
LogicalTableScan(table=[[scott, DEPT]])
LogicalTableScan(table=[[scott, EMP]])
{code}
The second conjunct only refers to columns from the right input, thus it could
be pushed to the right input. However, {{FilterJoinRule}} misses this
opportunity.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4467) Incorrect simplification for 'NaN' value
Jesus Camacho Rodriguez created CALCITE-4467:
Summary: Incorrect simplification for 'NaN' value
Key: CALCITE-4467
URL: https://issues.apache.org/jira/browse/CALCITE-4467
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
{{RexSimplify}} simplifies {{x = x}} to {{null or x is not null}} (similarly <=
and >=), and {{x != x}} to {{null and x is null}} (similarly < and >).
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L363
This may not be applicable in some cases. For instance, if the type of x is
floating-point, x could be 'NaN'. While some RDBMS consider 'NaN' = 'NaN'
(e.g., Postgres), some others consider 'NaN' != 'NaN' following the IEEE 754
standard. For the latest, the rewriting above will result in incorrect results.
I think we should simply ignore this simplification for floating-point type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3999) Simplify DialectPool implementation
Jesus Camacho Rodriguez created CALCITE-3999:
Summary: Simplify DialectPool implementation
Key: CALCITE-3999
URL: https://issues.apache.org/jira/browse/CALCITE-3999
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
JdbcUtils contains a pool to cache SqlDialect objects. Currently, it relies on
multiple maps and a synchronized {{get}} method. Although I am not very
familiar with that code, it seems the implementation could be made simpler and
more efficient by using a Guava cache. In addition, since we would not have a
single synchronized get method, multiple threads could concurrently create
dialects for distinct data sources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3982) FilterMergeRule can lead to AssertionError
Jesus Camacho Rodriguez created CALCITE-3982:
Summary: FilterMergeRule can lead to AssertionError
Key: CALCITE-3982
URL: https://issues.apache.org/jira/browse/CALCITE-3982
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
This could potentially happen since Filter creation has a check on whether the
expression is flat
([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
and Filter merge does not flatten an expression when it is created.
{noformat}
java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT
NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
at org.apache.calcite.rel.core.Filter.(Filter.java:74)
at
org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
at
org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
at
org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
at
org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3908) JoinCommuteRule does not update all input references in condition
Jesus Camacho Rodriguez created CALCITE-3908:
Summary: JoinCommuteRule does not update all input references in
condition
Key: CALCITE-3908
URL: https://issues.apache.org/jira/browse/CALCITE-3908
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
{{JoinCommuteRule}} swaps the inputs of a join. It relies on an internal class
{{VariableReplacer}} to update the references in the join condition. However,
this class does not implement {{RexShuttle}} and ends up ignoring some of the
references, e.g., those in {{RexFieldAccess}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3898) RelOptPredicateList may generate incorrect map of constant values
Jesus Camacho Rodriguez created CALCITE-3898:
Summary: RelOptPredicateList may generate incorrect map of
constant values
Key: CALCITE-3898
URL: https://issues.apache.org/jira/browse/CALCITE-3898
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
The method relies on {{RexUtil.predicateConstants}} which in turn calls
{{RexUtil.canAssignFrom}}. {{RexUtil.canAssignFrom}} is skipping any check on
precision and scale. I observed the error in Hive when two VARCHAR types with
different precision were given to the method, which was resulting on
considering the result of the narrowing cast as the value of the reference.
This lead to incorrect results.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
Jesus Camacho Rodriguez created CALCITE-3825: Summary: Split AbstractMaterializedViewRule into multiple classes Key: CALCITE-3825 URL: https://issues.apache.org/jira/browse/CALCITE-3825 Project: Calcite Issue Type: Improvement Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez AbstractMaterializedViewRule contains a materialized view-based rewriting algorithm that has been there for multiple releases and it is used by some engines relying in Calcite, e.g., Apache Hive. The main reason to have a single file/class for the rule was to make the logic self-contained instead of spreading it between multiple files from the onset, as it was experimental and we were not sure how far the implementation would go. In retrospective, we should have refactor that code sooner rather than later, since it makes very difficult to understand and maintain logic that is already complicated enough. This issue is to split AbstractMaterializedViewRule into multiple files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3189) Multiple fixes for Oracle SQL dialect
Jesus Camacho Rodriguez created CALCITE-3189:
Summary: Multiple fixes for Oracle SQL dialect
Key: CALCITE-3189
URL: https://issues.apache.org/jira/browse/CALCITE-3189
Project: Calcite
Issue Type: Bug
Affects Versions: 1.20.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Among others, it includes i) SQL translation support for custom types (e.g.
{{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length of {{VARCHAR}} type,
iii) creating datetime literals correctly, and iv) method to infer whether a
given data type is supported or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3066) RelToSqlConverter may incorrectly throw an AssertionError for some decimal literals
Jesus Camacho Rodriguez created CALCITE-3066:
Summary: RelToSqlConverter may incorrectly throw an AssertionError
for some decimal literals
Key: CALCITE-3066
URL: https://issues.apache.org/jira/browse/CALCITE-3066
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Issue can be reproduced adding the following query to {{RelToSqlConverterTest}}:
{code:sql}
select -0.000123 from "expense_fact";
{code}
{code}
Caused by: java.lang.AssertionError: -1.23E-8
at
org.apache.calcite.sql.SqlLiteral.createExactNumeric(SqlLiteral.java:872)
at
org.apache.calcite.rel.rel2sql.SqlImplementor$Context.toSql(SqlImplementor.java:502)
at
org.apache.calcite.rel.rel2sql.RelToSqlConverter.visit(RelToSqlConverter.java:186)
... 34 more
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2976) Improve materialized view rewriting coverage with disjunctive predicates
Jesus Camacho Rodriguez created CALCITE-2976:
Summary: Improve materialized view rewriting coverage with
disjunctive predicates
Key: CALCITE-2976
URL: https://issues.apache.org/jira/browse/CALCITE-2976
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
For instance, in the following case:
{code}
@Test public void testJoinAggregateMaterializationAggregateFuncs14() {
checkMaterialize(
"select \"empid\", \"emps\".\"name\", \"emps\".\"deptno\",
\"depts\".\"name\", "
+ "count(*) as c, sum(\"empid\") as s\n"
+ "from \"emps\" join \"depts\" using (\"deptno\")\n"
+ "where (\"depts\".\"name\" is not null and \"emps\".\"name\" = 'a')
or "
+ "(\"depts\".\"name\" is not null and \"emps\".\"name\" = 'b')\n"
+ "group by \"empid\", \"emps\".\"name\", \"depts\".\"name\",
\"emps\".\"deptno\"",
"select \"depts\".\"deptno\", sum(\"empid\") as s\n"
+ "from \"emps\" join \"depts\" using (\"deptno\")\n"
+ "where \"depts\".\"name\" is not null and \"emps\".\"name\" = 'a'\n"
+ "group by \"depts\".\"deptno\"",
HR_FKUK_MODEL,
CONTAINS_M0);
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2951) Support decorrelate subquery that has aggregate with grouping sets
Jesus Camacho Rodriguez created CALCITE-2951: Summary: Support decorrelate subquery that has aggregate with grouping sets Key: CALCITE-2951 URL: https://issues.apache.org/jira/browse/CALCITE-2951 Project: Calcite Issue Type: Sub-task Reporter: Jesus Camacho Rodriguez Assignee: Haisheng Yuan Fix For: 1.20.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2943) Materialized view rewriting logic calls getApplicableMaterializations each time the rule is triggered
Jesus Camacho Rodriguez created CALCITE-2943:
Summary: Materialized view rewriting logic calls
getApplicableMaterializations each time the rule is triggered
Key: CALCITE-2943
URL: https://issues.apache.org/jira/browse/CALCITE-2943
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Attachments: Screen Shot 2019-03-21 at 2.33.01 PM.png
{{RelOptMaterializations.getApplicableMaterializations}} is called each time
the rule is triggered.
{code:java}
...
// Obtain applicable (filtered) materializations
// TODO: Filtering of relevant materializations needs to be
// improved so we gather only materializations that might
// actually generate a valid rewriting.
final List applicableMaterializations =
RelOptMaterializations.getApplicableMaterializations(node,
materializations);
...
{code}
When I implemented the rule, I assumed (incorrectly) that
{{getApplicableMaterializations}} was a lightweight call and hence would help
discarding materialized views extracted from the planner quickly. It turns out
that the method can quickly become the most time consuming part of the rule
execution; I assume the method was just supposed to be used once per query.
!Screen Shot 2019-03-21 at 2.33.01 PM.png!
Since the prefiltering that we do right now is rather simple and we already
extract the tables used by queries and materializations within the rule, we can
just skip the materialization over there.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2942) Materialized view rewriting logic instantiates RelMetadataQuery each time the rule is triggered
Jesus Camacho Rodriguez created CALCITE-2942: Summary: Materialized view rewriting logic instantiates RelMetadataQuery each time the rule is triggered Key: CALCITE-2942 URL: https://issues.apache.org/jira/browse/CALCITE-2942 Project: Calcite Issue Type: Improvement Reporter: Jesus Camacho Rodriguez Performance penalty is similar to the one described in CALCITE-1812. An instance may be available in the cluster, hence we can use it; this is just an addendum to CALCITE-1812. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2883) HepPlanner subprogram may loop till getting out of memory
Jesus Camacho Rodriguez created CALCITE-2883:
Summary: HepPlanner subprogram may loop till getting out of memory
Key: CALCITE-2883
URL: https://issues.apache.org/jira/browse/CALCITE-2883
Project: Calcite
Issue Type: Bug
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Consider the following two hep programs.
Program 1:
{code}
final HepProgramBuilder programBuilder = new HepProgramBuilder();
programBuilder.addMatchOrder(HepMatchOrder.BOTTOM_UP);
programBuilder.addRuleInstance(JoinToMultiJoinRule.INSTANCE);
programBuilder.addRuleInstance(LoptOptimizeJoinRule.INSTANCE);
final HepProgram program = programBuilder.build();
{code}
Program 2:
{code}
final HepProgramBuilder programBuilder = new HepProgramBuilder();
final HepProgramBuilder subprogramBuilder = new HepProgramBuilder();
subprogramBuilder.addMatchOrder(HepMatchOrder.BOTTOM_UP);
subprogramBuilder.addRuleInstance(JoinToMultiJoinRule.INSTANCE);
subprogramBuilder.addRuleInstance(LoptOptimizeJoinRule.INSTANCE);
programBuilder.addSubprogram(subprogramBuilder.build());
final HepProgram program = programBuilder.build();
{code}
I would expect both programs to behave similarly. However, program 2 will loop
indefinitely. The reason is that {{HepPlanner}} subprogram execution loops if
subprogram generates any new expression.
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/hep/HepPlanner.java#L339
This does not seem right since planner can control exiting the program (and
thus, subprogram) depending on its own internal state and configuration
properties, e.g., match limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2858) JSON writer and reader should include type information for literals
Jesus Camacho Rodriguez created CALCITE-2858: Summary: JSON writer and reader should include type information for literals Key: CALCITE-2858 URL: https://issues.apache.org/jira/browse/CALCITE-2858 Project: Calcite Issue Type: Bug Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.19.0 JSON writer does not include information about literals type (unless the literal is null). This can lead to some ambiguity when parsing the plan back into RelNodes by the JSON reader. This issue is to include this information in the writer, and to be able to parse it back in the reader. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2740) Add tests for SqlFunction.supportsFunction method
Jesus Camacho Rodriguez created CALCITE-2740: Summary: Add tests for SqlFunction.supportsFunction method Key: CALCITE-2740 URL: https://issues.apache.org/jira/browse/CALCITE-2740 Project: Calcite Issue Type: Test Components: jdbc-adapter Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Method is used in Hive, but it is not used by Calcite at all. At the very least, we should add some unit tests to check that the method is working as expected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2733) Use catalog and schema from JDBC connection string to retrieve tables if specified
Jesus Camacho Rodriguez created CALCITE-2733: Summary: Use catalog and schema from JDBC connection string to retrieve tables if specified Key: CALCITE-2733 URL: https://issues.apache.org/jira/browse/CALCITE-2733 Project: Calcite Issue Type: Improvement Components: jdbc-adapter Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez >From JDBC 4.1, catalog and schema can be retrieved from the connection object. >When we retrieve the table objects using the JDBC connection, I believe we >could try to get catalog and schema from connection object if they have not >been specified by user. If they are not in the connection object either, null >will be passed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2732) Upgrade postgresql driver version
Jesus Camacho Rodriguez created CALCITE-2732:
Summary: Upgrade postgresql driver version
Key: CALCITE-2732
URL: https://issues.apache.org/jira/browse/CALCITE-2732
Project: Calcite
Issue Type: Test
Components: jdbc-adapter
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
We are still using {{9.3-1102-jdbc3}} version. Not sure if anyone has run the
compatibility tests in calcite-test-dataset with Postgresql recently, but I get
an java.lang.AbstractMethodError message for several of them. We can move to
the more recent {{9.3-1104-jdbc41}} (I verified that this fixes the issue).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2713) JDBC adapter may generate casts on PostgreSQL for VARCHAR type exceeding max length
Jesus Camacho Rodriguez created CALCITE-2713:
Summary: JDBC adapter may generate casts on PostgreSQL for VARCHAR
type exceeding max length
Key: CALCITE-2713
URL: https://issues.apache.org/jira/browse/CALCITE-2713
Project: Calcite
Issue Type: Bug
Components: jdbc-adapter
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Varchar length in PostgreSQL cannot exceed 10485760, however Calcite may
generate a cast with length larger than that number, resulting in an exception.
{noformat}
org.postgresql.util.PSQLException: ERROR: length for type varchar cannot exceed
10485760
{noformat}
>From {{htup_details.h}} in postgresql:
{noformat}
* MaxAttrSize is a somewhat arbitrary upper limit on the declared size of
* data fields of char(n) and similar types. It need not have anything
* directly to do with the *actual* upper limit of varlena values, which
* is currently 1Gb (see TOAST structures in postgres.h). I've set it
* at 10Mb which seems like a reasonable number --- tgl 8/6/00. */
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2673) SqlDialect supports pushing of all functions by default
Jesus Camacho Rodriguez created CALCITE-2673: Summary: SqlDialect supports pushing of all functions by default Key: CALCITE-2673 URL: https://issues.apache.org/jira/browse/CALCITE-2673 Project: Calcite Issue Type: Improvement Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.18.0 SqlDialect contains a 'supportsFunction' that can be used by rules to know whether a certain function is supported in the given dialect, e.g., to choose whether to push a Filter expression to JDBC, etc. The default implementation of 'supportsFunction' always returns true. I believe a better idea would be to support in the default implementation for the method the most common SQL functions. Then each dialect can override that behavior and expand/limit the supported functions, e.g., JethroDataDialect already does that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2669) RelMdTableReferences should check whether references inferred from input are null for Union/Join operators
Jesus Camacho Rodriguez created CALCITE-2669: Summary: RelMdTableReferences should check whether references inferred from input are null for Union/Join operators Key: CALCITE-2669 URL: https://issues.apache.org/jira/browse/CALCITE-2669 Project: Calcite Issue Type: Bug Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.18.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2668) Support for left/right outer join in RelMdExpressionLineage
Jesus Camacho Rodriguez created CALCITE-2668: Summary: Support for left/right outer join in RelMdExpressionLineage Key: CALCITE-2668 URL: https://issues.apache.org/jira/browse/CALCITE-2668 Project: Calcite Issue Type: Improvement Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Currently, we bail out in the metadata provider if join operator is not of inner type. For left, right outer joins, we could track expressions generated from columns from the left, right inputs, respectively. In addition, if any of the columns in the join cannot be backtracked, we could still backtrack the origin of an expression if it is not using that column, and currently we bail out too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2622) RexFieldCollation toString method is not deterministic
Jesus Camacho Rodriguez created CALCITE-2622: Summary: RexFieldCollation toString method is not deterministic Key: CALCITE-2622 URL: https://issues.apache.org/jira/browse/CALCITE-2622 Project: Calcite Issue Type: Bug Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez It iterates over a set to print its flags. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2465) Enable use of materialized views for any planner
Jesus Camacho Rodriguez created CALCITE-2465: Summary: Enable use of materialized views for any planner Key: CALCITE-2465 URL: https://issues.apache.org/jira/browse/CALCITE-2465 Project: Calcite Issue Type: Improvement Affects Versions: 1.17.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Currently it is only supported in VolcanoPlanner. Using HepPlanner can be useful for debugging purposes. We only need to create/override the relevant method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2387) Fix for date/timestamp cast expressions in Druid adapter
Jesus Camacho Rodriguez created CALCITE-2387: Summary: Fix for date/timestamp cast expressions in Druid adapter Key: CALCITE-2387 URL: https://issues.apache.org/jira/browse/CALCITE-2387 Project: Calcite Issue Type: Bug Components: druid Affects Versions: 1.17.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.17.0 Follow-up for CALCITE-2286. Timezone should be set correctly by Druid when there is a cast for timestamp type vs timestamp with local time zone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2334) Extend simplification of expressions with CEIL function over date types
Jesus Camacho Rodriguez created CALCITE-2334:
Summary: Extend simplification of expressions with CEIL function
over date types
Key: CALCITE-2334
URL: https://issues.apache.org/jira/browse/CALCITE-2334
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.17.0
Reporter: Jesus Camacho Rodriguez
Assignee: Julian Hyde
CALCITE-2332 disables simplification of CEIL function due to correctness issues.
{{FLOOR}} and {{CEIL}} cannot be handled with the same logic. For instance,
{{CEIL(CEIL(x TO HOUR) TO YEAR)}} cannot be simplified, e.g., '2011-12-31
23:59:59', while {{FLOOR(FLOOR(x TO HOUR) TO YEAR)}} can be simplified. Hence,
we need new logic to enable simplification of CEIL function on date types.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2300) Improve messages shown in materialized view-based rewriting algorithm
Jesus Camacho Rodriguez created CALCITE-2300: Summary: Improve messages shown in materialized view-based rewriting algorithm Key: CALCITE-2300 URL: https://issues.apache.org/jira/browse/CALCITE-2300 Project: Calcite Issue Type: Improvement Affects Versions: 1.16.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Currently, we do not log or expose any information on why a specific rewriting based on a materialized view was not triggered. Probably we should do that at a very high level of granularity, for instance using the logger TRACE level. This will help 1) debugging possible missing rewriting options, and 2) possibly providing feedback to user if needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2286) Support timestamp type for Druid adapter
Jesus Camacho Rodriguez created CALCITE-2286:
Summary: Support timestamp type for Druid adapter
Key: CALCITE-2286
URL: https://issues.apache.org/jira/browse/CALCITE-2286
Project: Calcite
Issue Type: Improvement
Components: druid
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.17.0
CALCITE-1947 replaced {{timestamp}} type for {{timestamp with local time zone}}
in Druid adapter. This issue aims at supporting both types for Druid-backed
tables and let the user decide the type/semantics they want to use.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2277) Skip SemiJoin operator in materialized view-based rewriting algorithm
Jesus Camacho Rodriguez created CALCITE-2277:
Summary: Skip SemiJoin operator in materialized view-based
rewriting algorithm
Key: CALCITE-2277
URL: https://issues.apache.org/jira/browse/CALCITE-2277
Project: Calcite
Issue Type: Bug
Affects Versions: 1.16.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.17.0
{{AbstractMaterializedViewRule}} is not recognizing the difference between
SemiJoin and Join operator, since SemiJoin inherits from Join. This can lead to
incorrect rewriting and errors in the rewriting logic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2232) Assertion error on AggregatePullUpConstantsRule
Jesus Camacho Rodriguez created CALCITE-2232:
Summary: Assertion error on AggregatePullUpConstantsRule
Key: CALCITE-2232
URL: https://issues.apache.org/jira/browse/CALCITE-2232
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.16.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.17.0
Executing the following query:
{code:sql}
select ename, sal
from (select '1', ename, sal from emp where ename = 'John') subq
group by ename, sal;
{code}
results in the following error:
{code}
java.lang.AssertionError: Cannot add expression of different type to set:
set type is RecordType(VARCHAR(20) CHARACTER SET "ISO-8859-1" COLLATE
"ISO-8859-1$en_US$primary" NOT NULL ENAME, INTEGER NOT NULL SAL) NOT NULL
expression type is RecordType(INTEGER NOT NULL ENAME, INTEGER NOT NULL SAL) NOT
NULL
set is rel#21:LogicalAggregate(input=HepRelVertex#20,group={1, 5})
expression is LogicalProject#24
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2212) Avatica - Enforce Java version via maven-enforcer-plugin
Jesus Camacho Rodriguez created CALCITE-2212: Summary: Avatica - Enforce Java version via maven-enforcer-plugin Key: CALCITE-2212 URL: https://issues.apache.org/jira/browse/CALCITE-2212 Project: Calcite Issue Type: Task Components: avatica, core Reporter: Josh Elser Assignee: Kevin Risden Fix For: 1.16.0 Now that jdk7 support has been dropped, we should add some logic to the build to fail obviously when a version of Java is used that we don't support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2192) RelBuilder might skip creation of Aggregate though it has a column pruning effect
Jesus Camacho Rodriguez created CALCITE-2192:
Summary: RelBuilder might skip creation of Aggregate though it has
a column pruning effect
Key: CALCITE-2192
URL: https://issues.apache.org/jira/browse/CALCITE-2192
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.16.0
Issue can be reproduced with following test:
{code:java}
@Test public void testAggregate3() {
final RelBuilder builder = RelBuilder.create(config().build());
RelNode root =
builder.scan("EMP")
.aggregate(
builder.groupKey(builder.field(1)),
builder.aggregateCall(SqlStdOperatorTable.COUNT, false, false,
null, "C"))
.aggregate(
builder.groupKey(builder.field(0)))
.build();
assertThat(str(root),
is(""
+ "LogicalProject(ENAME=[$0])\n"
+ " LogicalAggregate(group=[{1}], C=[COUNT()])\n"
+ "LogicalTableScan(table=[[scott, EMP]])\n"));
}
{code}
Without fix, builder will generate following plan, which contains an
unnecessary field (in Hive, this results in an assertion error in
RelFieldTrimmer):
{code:java}
LogicalAggregate(group=[{1}], C=[COUNT()])
LogicalTableScan(table=[[scott, EMP]])
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2190) Extend SubstitutionVisitor.splitFilter to cover different order of operands
Jesus Camacho Rodriguez created CALCITE-2190:
Summary: Extend SubstitutionVisitor.splitFilter to cover different
order of operands
Key: CALCITE-2190
URL: https://issues.apache.org/jira/browse/CALCITE-2190
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
{{SubstitutionVisitor.splitFilter}} does structural comparison to identify
relevant predicates. The method could sort the operands for some expressions in
a deterministic way to maximize possible matches.
For instance, currently this example yields correct results:
{code}
condition: x = 1 or y = 2
target:y = 2 or x = 1
-> residue: true
{code}
However, the following equivalent example fails:
{code}
condition: x = 1 or y = 2
target:y = 2 or 1 = x
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2189) RelMdAllPredicates fast bail out creates mismatch with RelMdTableReferences
Jesus Camacho Rodriguez created CALCITE-2189:
Summary: RelMdAllPredicates fast bail out creates mismatch with
RelMdTableReferences
Key: CALCITE-2189
URL: https://issues.apache.org/jira/browse/CALCITE-2189
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.16.0
The idea behind the metadata providers introduced in CALCITE-1682 is that we
can identify the lineage of the expressions. If we bypass assigning a unique
identifier for the table in RelMdAllPredicates because there are no predicates
on those tables, then we will end up referencing wrong tables.
E.g.
{code}
select x.sal from
(select a.deptno, c.sal from (select * from emp limit 7) as a
cross join (select * from dept limit 1) as b
cross join (select * from emp where empno = 5 limit 2) as c) as x;
{code}
Table refs: {{[[CATALOG, SALES, DEPT].#0, [CATALOG, SALES, EMP].#0, [CATALOG,
SALES, EMP].#1]}}
Extracted predicate without fix (wrong): {{=([CATALOG, SALES, EMP].#0.$0, 5)}}
Extracted predicate with fix (correct): {{=([CATALOG, SALES, EMP].#1.$0, 5)}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2181) Release Calcite 1.16.0
Jesus Camacho Rodriguez created CALCITE-2181: Summary: Release Calcite 1.16.0 Key: CALCITE-2181 URL: https://issues.apache.org/jira/browse/CALCITE-2181 Project: Calcite Issue Type: Task Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Release Calcite 1.16.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2179) General improvements for materialized view rewriting rule
Jesus Camacho Rodriguez created CALCITE-2179:
Summary: General improvements for materialized view rewriting rule
Key: CALCITE-2179
URL: https://issues.apache.org/jira/browse/CALCITE-2179
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.16.0
This issue is for extending {{AbstractMaterializedViewRule}} rule:
- Support for rolling up date nodes. For instance, rewrite in the following
case:
{code}
Materialization:
select "empid", floor(cast('1997-01-20' as timestamp) to month), count(*) + 1
as c, sum("empid") as s
from "emps" group by "empid", floor(cast('1997-01-20' as timestamp) to month);
Query:
select floor(cast('1997-01-20' as timestamp) to year), sum("empid") as s
from "emps" group by floor(cast('1997-01-20' as timestamp) to year);
{code}
- Add flag to enable/disable fast bail out for joins. By default it is true,
and thus, we were only creating the rewriting in the minimal subtree of plan
operators. For instance:
{code}
View: (A JOIN B) JOIN C
Query: (((A JOIN B) JOIN D) JOIN C) JOIN E
{code}
We produce it at:
{code}
((A JOIN B) JOIN D) JOIN C
{code}
But not at:
{code}
(((A JOIN B) JOIN D) JOIN C) JOIN E
{code}
This is important when the rule is used with the Volcano planner together with
other rules, e.g. join reordering, as it prevents that the search space grows
unnecessarily. However, if we use the rewriting rule in isolation, fast bail
out can lead to missing rewriting opportunities (e.g. for bushy join trees).
- Possibility to provide a HepProgram to optimize query branch in union
rewritings. Note that when we produce a partial rewriting with a Union, the
branch that will execute the (partial) query can be fully rewritten so we can
add the compensation predicate. (We cannot do the same for views because the
expression might not be computable if the needed subexpressions are not
available in the view output). If we use Volcano with a determined set of
rules, this might not be needed, hence providing this program is optional.
- Multiple small fixes discovered while testing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2178) Extend expression simplifier to work on datetime FLOOR functions
Jesus Camacho Rodriguez created CALCITE-2178:
Summary: Extend expression simplifier to work on datetime FLOOR
functions
Key: CALCITE-2178
URL: https://issues.apache.org/jira/browse/CALCITE-2178
Project: Calcite
Issue Type: Improvement
Components: core
Environment: Extend expression simplifier to support:
{code}
FLOOR(FLOOR(CAST('2010-01-10 00:00:00' AS TIMESTAMP) TO HOUR) TO DAY) =>
FLOOR(CAST('2010-01-10 00:00:00' AS TIMESTAMP) TO DAY)
{code}
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.16.0
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2137) Materialized view rewriting not being triggered for some join queries
Jesus Camacho Rodriguez created CALCITE-2137:
Summary: Materialized view rewriting not being triggered for some
join queries
Key: CALCITE-2137
URL: https://issues.apache.org/jira/browse/CALCITE-2137
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.16.0
The issue has to do with the column equivalences mapping for joins with
equality predicates for columns that are output by the query or subquery
(basically, there is a bug and we do not apply mapping). This results in
missing rewriting opportunities as the top expression cannot be mapped from the
query to the view. It can be reproduced with the following MV and query in
{{MaterializationTest.java}}:
MV:
{code}
select *
from "emps"
join "dependents" using ("empid");
{code}
Query:
{code}
select "emps"."empid", "dependents"."empid", "emps"."deptno"
from "emps"
join "dependents" using ("empid")
join "depts" "a" on ("emps"."deptno"="a"."deptno")
where "emps"."name" = 'Bill';
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2074) Simplification of AND/OR expressions yields wrong results
Jesus Camacho Rodriguez created CALCITE-2074:
Summary: Simplification of AND/OR expressions yields wrong results
Key: CALCITE-2074
URL: https://issues.apache.org/jira/browse/CALCITE-2074
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Julian Hyde
Discovered while testing 1.15.0 RC0 with Hive. It seems this regression was
introduced by CALCITE-1995.
Consider the following query:
{code}
select * from (
select a.*,b.d d1,c.d d2 from
t1 a join t2 b on (a.id1 = b.id)
join t2 c on (a.id2 = b.id) where b.d <= 1 and c.d <= 1
) z where d1 > 1 or d2 > 1
{code}
We end up generating the following plan:
{code}
HiveProject(id1=[$0], id2=[$1], d1=[$3], d2=[$4])
HiveJoin(condition=[OR(=($3, 1), =($4, 1))], joinType=[inner],
algorithm=[none], cost=[not available])
HiveJoin(condition=[AND(=($0, $2), =($1, $2))], joinType=[inner],
algorithm=[none], cost=[not available])
HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))])
HiveProject(id1=[$0], id2=[$1])
HiveTableScan(table=[[default.t1]], table:alias=[a])
HiveFilter(condition=[AND(<=($1, 1), IS NOT NULL($0))])
HiveProject(id=[$0], d=[$1])
HiveTableScan(table=[[default.t2]], table:alias=[b])
HiveFilter(condition=[<=($0, 1)])
HiveProject(d=[$1])
HiveTableScan(table=[[default.t2]], table:alias=[c])
{code}
Observe that the condition in the top join is not correct.
I can reproduce this in {{RexProgramTest.simplifyFilter}} with the following
example:
{code}
// condition "a > 5 or b > 5"
// with pre-condition "a <= 5 and b <= 5"
// should yield "false" but yields "a = 5 or b = 5"
checkSimplifyFilter(or(gt(aRef, literal5),gt(bRef, literal5)),
RelOptPredicateList.of(rexBuilder,
ImmutableList.of(le(aRef, literal5), le(bRef, literal5))),
"false");
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CALCITE-2062) Prevent creating an Aggregate with grouping columns that are not present in grouping sets
Jesus Camacho Rodriguez created CALCITE-2062: Summary: Prevent creating an Aggregate with grouping columns that are not present in grouping sets Key: CALCITE-2062 URL: https://issues.apache.org/jira/browse/CALCITE-2062 Project: Calcite Issue Type: Bug Reporter: Jesus Camacho Rodriguez Assignee: Julian Hyde Reject an Aggregate if there are columns that are not in any of the grouping sets. See proposed fix in https://github.com/julianhyde/calcite/tree/2051-minimal-groupSet. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CALCITE-2052) Remove SQL code style from MV documentation
Jesus Camacho Rodriguez created CALCITE-2052: Summary: Remove SQL code style from MV documentation Key: CALCITE-2052 URL: https://issues.apache.org/jira/browse/CALCITE-2052 Project: Calcite Issue Type: Improvement Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.15.0 As it is not rendered properly in website. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CALCITE-2051) Rules using Aggregate might check for simple grouping sets incorrectly
Jesus Camacho Rodriguez created CALCITE-2051:
Summary: Rules using Aggregate might check for simple grouping
sets incorrectly
Key: CALCITE-2051
URL: https://issues.apache.org/jira/browse/CALCITE-2051
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Priority: Critical
Fix For: 1.15.0
CALCITE-1069 removed the indicator columns for Aggregate operators. In some
places, the indicator boolean check was replaced by the following check:
{{aggregate.getGroupSets().size() > 1}}. However, that check is incomplete, it
should have been replace by {{aggregate.getGroupType() == Group.SIMPLE}}.
For instance :
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/AggregateProjectMergeRule.java#L91
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CALCITE-2012) Replace LocalInterval by Interval in Druid adapter
Jesus Camacho Rodriguez created CALCITE-2012: Summary: Replace LocalInterval by Interval in Druid adapter Key: CALCITE-2012 URL: https://issues.apache.org/jira/browse/CALCITE-2012 Project: Calcite Issue Type: Task Components: druid Affects Versions: 1.14.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.15.0 CALCITE-1617 introduced LocalInterval as a proper way to close the gap between the semantics of SQL timestamp type and Druid instants. After that, CALCITE-1947 introduced 'timestamp with local time zone' type in Calcite and mapped the Druid time column to this type. Thus, we do not need anymore the LocalInterval class and we can use Joda Interval, since the column represents an Instant rather than a LocalDateTime. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CALCITE-1983) Push EQUALS and NOT EQUALS operations with numeric cast on dimensions
Jesus Camacho Rodriguez created CALCITE-1983:
Summary: Push EQUALS and NOT EQUALS operations with numeric cast
on dimensions
Key: CALCITE-1983
URL: https://issues.apache.org/jira/browse/CALCITE-1983
Project: Calcite
Issue Type: Improvement
Components: druid
Affects Versions: 1.15.0
Reporter: slim bouguerra
Assignee: Jesus Camacho Rodriguez
Fix For: 1.15.0
For instance, the following query should be pushed as a time-series query with
filters.
{code}
use ssb_druid;
Time taken: 0.229 seconds, Fetched: 24 row(s)
hive>
> explain select
> sum(discounted_price)
> from
> ssb_druid_day
> where
> lo_quantity = 25 ;
OK
Plan optimized by CBO.
Vertex dependency in root stage
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Stage-0
Fetch Operator
limit:-1
Stage-1
Reducer 2 vectorized, llap
File Output Operator [FS_14]
Group By Operator [GBY_13] (rows=1 width=8)
Output:["_col0"],aggregations:["sum(VALUE._col0)"]
<-Map 1 [SIMPLE_EDGE] vectorized, llap
SHUFFLE [RS_12]
Group By Operator [GBY_11] (rows=1 width=8)
Output:["_col0"],aggregations:["sum(discounted_price)"]
Select Operator [SEL_10] (rows=1 width=0)
Output:["discounted_price"]
Filter Operator [FIL_9] (rows=1 width=0)
predicate:(UDFToDouble(lo_quantity) = 25.0)
TableScan [TS_0] (rows=589709 width=0)
ssb_druid@ssb_druid_day,ssb_druid_day,Tbl:PARTIAL,Col:NONE,Output:["lo_quantity","discounted_price"],properties:{"druid.query.json":"{\"queryType\":\"select\",\"dataSource\":\"ssb_druid_day\",\"descending\":false,\"intervals\":[\"1900-01-01T00:00:00.000/3000-01-01T00:00:00.000\"],\"dimensions\":[\"c_city\",\"c_nation\",\"c_region\",\"d_weeknuminyear\",\"d_year\",\"d_yearmonth\",\"d_yearmonthnum\",\"lo_discount\",\"lo_quantity\",\"p_brand1\",\"p_category\",\"p_mfgr\",\"s_city\",\"s_nation\",\"s_region\"],\"metrics\":[\"lo_revenue\",\"discounted_price\",\"net_revenue\"],\"granularity\":\"all\",\"pagingSpec\":{\"threshold\":16384,\"fromNext\":true},\"context\":{\"druid.query.fetch\":false}}","druid.query.type":"select"}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CALCITE-1982) NPE simplifying range expressions when literal value is null
Jesus Camacho Rodriguez created CALCITE-1982:
Summary: NPE simplifying range expressions when literal value is
null
Key: CALCITE-1982
URL: https://issues.apache.org/jira/browse/CALCITE-1982
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.14.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.14.0
The problem is that we do not check for {{null}} literal values before going
into the method that tries to simplify the ranges.
{code}
java.lang.NullPointerException: null
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
~[guava-14.0.1.jar:?]
at com.google.common.collect.Cut$AboveValue.(Cut.java:301)
~[guava-14.0.1.jar:?]
at com.google.common.collect.Cut.aboveValue(Cut.java:296)
~[guava-14.0.1.jar:?]
at com.google.common.collect.Range.atMost(Range.java:249)
~[guava-14.0.1.jar:?]
at
org.apache.calcite.rex.RexSimplify.processRange(RexSimplify.java:849)
~[calcite-core-1.13.0.jar:1.13.0]
at
org.apache.calcite.rex.RexSimplify.simplifyAnd2ForUnknownAsFalse(RexSimplify.java:668)
~[calcite-core-1.13.0.jar:1.13.0]
at
org.apache.calcite.rex.RexSimplify.simplifyAnds(RexSimplify.java:226)
~[calcite-core-1.13.0.jar:1.13.0]
at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:811)
~[calcite-core-1.13.0.jar:1.13.0]
at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:801)
~[calcite-core-1.13.0.jar:1.13.0]
...
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CALCITE-1859) NPE in validate method of VolcanoPlanner
Jesus Camacho Rodriguez created CALCITE-1859:
Summary: NPE in validate method of VolcanoPlanner
Key: CALCITE-1859
URL: https://issues.apache.org/jira/browse/CALCITE-1859
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.13.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Priority: Critical
Fix For: 1.14.0
CALCITE-1812 introduced the following line in {{validate}} method in
VolcanoPlanner:
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoPlanner.java#L891
{code}
final RelMetadataQuery mq = root.getCluster().getMetadataQuery();
{code}
{{validate}} might be called as part of the {{setRoot}} logic before _root_ is
set, thus we are hitting a NPE. Workaround was easy as {{validate}} is only
called in logging DEBUG level (I guess that is why we did not see this issue
before), but this JIRA will fix the issue by retrieving the RelMetadataQuery in
_validate_ only when needed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CALCITE-1808) JaninoRelMetadataProvider loading cache might cause OOM error
Jesus Camacho Rodriguez created CALCITE-1808:
Summary: JaninoRelMetadataProvider loading cache might cause OOM
error
Key: CALCITE-1808
URL: https://issues.apache.org/jira/browse/CALCITE-1808
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jagruti Varia
Assignee: Julian Hyde
JaninoRelMetadataProvider has a static handler cache with size increasing over
time for long running HS2, ending up causing OOM errors.
{code:java}
/** Cache of pre-generated handlers by provider and kind of metadata.
* For the cache to be effective, providers should implement identity
* correctly. */
private static final LoadingCache HANDLERS =
CacheBuilder.newBuilder().build(
new CacheLoader() {
public MetadataHandler load(@Nonnull Key key) {
//noinspection unchecked
return load3(key.def, key.provider.handlers(key.def),
key.relClasses);
}
});
...
/** Key for the cache. */
private static class Key {
public final MetadataDef def;
public final RelMetadataProvider provider;
public final ImmutableList relClasses;
...
{code}
The lifecycle for providers is per query and we have multiple providers
instantiated on the lifecycle of a query. The entries are retained in the cache
even when query planning has finished.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CALCITE-1802) Add post-aggregation step for Union in materialized view rewriting
Jesus Camacho Rodriguez created CALCITE-1802:
Summary: Add post-aggregation step for Union in materialized view
rewriting
Key: CALCITE-1802
URL: https://issues.apache.org/jira/browse/CALCITE-1802
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.13.0
Follow-up on CALCITE-1795.
Rewriting for Aggregate queries needs a post-aggregation step that is not
currently added.
Query:
{code:sql}
SELECT empid, deptname, SUM(salary) AS s
FROM emps
JOIN depts ON (emps.deptno = depts.deptno)
WHERE salary > 1
GROUP BY empid, deptname;
{code}
Materialized view definition:
{code:sql}
SELECT empid, deptname, SUM(salary) AS s
FROM emps
JOIN depts ON (emps.deptno = depts.deptno)
WHERE salary > 12000
GROUP BY empid, deptname;
{code}
Rewriting:
{code:sql}
SELECT empid, deptname, SUM(s)
FROM (
SELECT empid, deptname, s
FROM mv
UNION ALL
SELECT empid, deptname, SUM(salary) AS s
FROM emps
JOIN depts ON (emps.deptno = depts.deptno)
WHERE salary > 1 AND salary <= 12000
GROUP BY empid, deptname);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CALCITE-1797) Support view partial rewriting in aggregate materialized view rewriting
Jesus Camacho Rodriguez created CALCITE-1797: Summary: Support view partial rewriting in aggregate materialized view rewriting Key: CALCITE-1797 URL: https://issues.apache.org/jira/browse/CALCITE-1797 Project: Calcite Issue Type: Improvement Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.13.0 Simple extension for AbstractMaterializedViewRule similar to CALCITE-1791, however for views/queries rooted with an Aggregate node. In this case, we need to rely on FK-UK relationship to check whether the missing tables are just appending columns to the view result or even filtering the result, but not changing their multiplicity. If they meet these requirements, we can join the missing tables to the view and the view plan (that will be modified accordingly), and let the rewriting algorithm work the same way and enforce any missing predicate. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CALCITE-1796) Update materialized views documentation
Jesus Camacho Rodriguez created CALCITE-1796: Summary: Update materialized views documentation Key: CALCITE-1796 URL: https://issues.apache.org/jira/browse/CALCITE-1796 Project: Calcite Issue Type: Improvement Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.13.0 After CALCITE-1731 and follow-up works such as CALCITE-1791, we need to update the documentation to reflect clearly the coverage of the materialized view rewriting algorithm, as it is already larger than the rewriting coverage of the paper used for the baseline algorithm. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CALCITE-1795) Extend materialized view rewriting to produce rewritings using Union operators
Jesus Camacho Rodriguez created CALCITE-1795: Summary: Extend materialized view rewriting to produce rewritings using Union operators Key: CALCITE-1795 URL: https://issues.apache.org/jira/browse/CALCITE-1795 Project: Calcite Issue Type: Improvement Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.13.0 Extend join/aggregate materialized rewriting rule to produce rewritings using Union operators, e.g., a given query could be partially answered from the MV (year = 2014) and from the query (not(year=2014)). If the MV is stored e.g. in Druid, this rewriting might be beneficial. As with the other rewritings, decision on whether to finally use the rewriting is cost-based. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CALCITE-1791) Support view partial rewriting in join materialized view rewriting
Jesus Camacho Rodriguez created CALCITE-1791:
Summary: Support view partial rewriting in join materialized view
rewriting
Key: CALCITE-1791
URL: https://issues.apache.org/jira/browse/CALCITE-1791
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.13.0
Simple extension for {{AbstractMaterializedViewRule}} to support case when view
contains a subset of the tables of the query and tables are joined in different
order in the query and view plans.
For instance:
View (m0): {{(A JOIN B) JOIN C}}
Query: {{(((A JOIN B) JOIN D) JOIN C) JOIN E}}
MV rewriting: {{((m0 JOIN D) JOIN E)}}
Basically, once we have found the missing tables, we add them to the view and
view plan. Then the rewriting algorithm works the same way and will enforce any
predicate that is in the query and not in the view.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CALCITE-1763) Recognize lossless casts in join/aggregate materialized view rewriting rule
Jesus Camacho Rodriguez created CALCITE-1763: Summary: Recognize lossless casts in join/aggregate materialized view rewriting rule Key: CALCITE-1763 URL: https://issues.apache.org/jira/browse/CALCITE-1763 Project: Calcite Issue Type: Improvement Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.13.0 Integrate CALCITE-1760 with rewriting rule so we can cover additional cases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CALCITE-1749) Push filter conditions partially into Druid
Jesus Camacho Rodriguez created CALCITE-1749: Summary: Push filter conditions partially into Druid Key: CALCITE-1749 URL: https://issues.apache.org/jira/browse/CALCITE-1749 Project: Calcite Issue Type: Improvement Components: druid Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.13.0 When we cannot push the full filter predicate into DruidQuery, we should push it partially and leave the rest of the predicate on top of it. This issue extends the logic of the _DruidFilterRule_ to do that. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CALCITE-1731) Rewriting of queries using materialized views with joins and aggregates
Jesus Camacho Rodriguez created CALCITE-1731: Summary: Rewriting of queries using materialized views with joins and aggregates Key: CALCITE-1731 URL: https://issues.apache.org/jira/browse/CALCITE-1731 Project: Calcite Issue Type: New Feature Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.13.0 The idea is still to build a rewriting approach similar to: ftp://ftp.cse.buffalo.edu/users/azhang/disc/SIGMOD/pdf-files/331/202-optimizing.pdf I tried to build on CALCITE-1389 work. However, finally I ended up creating a new alternative rule. The main reason is that I wanted to follow the paper more closely and not rely on triggering rules within the MV rewriting to find whether expressions are equivalent. Instead, we extract information from the query plan and the MVs plans using the new metadata providers proposed in CALCITE-1682, and then we use that information to validate and execute the rewriting. I also implemented new unifying/rewriting logic within the rule, since existing unifying rules for aggregates were assuming that aggregate inputs in the query and the MV needed to be equivalent (same Volcano node). That condition can be relaxed because we verify in the rule, by using the new metadata providers as stated above, that the result for the query is contained within the MV. I added multiple tests, but any feedback pointing to new tests that could be added to check correctness/coverage is welcome. Algorithm can trigger multiple rewritings for the same query node. In addition, support for multiple usages of tables in query/MVs is supported. A few extensions that will follow this issue: * Extend logic to filter relevant MVs for a given query node, so approach is scalable as number of MVs grows. * Produce rewritings using Union operators, e.g., a given query could be partially answered from the MV (_year = 2014_) and from the query (_not(year=2014)_). If the MV is stored e.g. in Druid, this rewriting might be beneficial. As with the other rewritings, decision on whether to finally use the rewriting should be cost-based. * Currently query and MV must use the same tables. This logic can be extended: - Firstly, if query uses an additional table than MV, we can produce a rewriting that joins the MV with that additional table (given that join keys are available in the MV and we can compute all output columns). - Second, if MV uses more tables than the query, we can recognize the cardinality preserving joins to just project columns out of the MV and use it in the rewriting. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CALCITE-1724) Wrong comparison for floats/double type in Druid
Jesus Camacho Rodriguez created CALCITE-1724:
Summary: Wrong comparison for floats/double type in Druid
Key: CALCITE-1724
URL: https://issues.apache.org/jira/browse/CALCITE-1724
Project: Calcite
Issue Type: Bug
Components: druid
Reporter: slim bouguerra
Assignee: slim bouguerra
Fix For: 1.13.0
Follow-up on CALCITE-1722.
{quote}
The second issue (Minor) is to use {code}ordering{code} instead of
Alphanumeric, this is needed when comparing floats/doubles.
{quote}
It needs Calcite to move to Druid 0.9.2.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CALCITE-1723) Match DruidProjectFilterTransposeRule against DruidQuery
Jesus Camacho Rodriguez created CALCITE-1723: Summary: Match DruidProjectFilterTransposeRule against DruidQuery Key: CALCITE-1723 URL: https://issues.apache.org/jira/browse/CALCITE-1723 Project: Calcite Issue Type: Bug Reporter: Jesus Camacho Rodriguez Assignee: Nishant Bangarwa Priority: Minor Fix For: 1.13.0 Follow-up CALCITE-1683. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CALCITE-1661) Recognize aggregation function types as FRACTIONAL instead of DOUBLE
Jesus Camacho Rodriguez created CALCITE-1661:
Summary: Recognize aggregation function types as FRACTIONAL
instead of DOUBLE
Key: CALCITE-1661
URL: https://issues.apache.org/jira/browse/CALCITE-1661
Project: Calcite
Issue Type: Bug
Components: druid
Affects Versions: 1.12.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Fix For: 1.12.0
Currently, whether to use fractional or integer aggregations is based on
following code (L699 in DruidQuery.java).
{code}
final boolean b = aggCall.getType().getSqlTypeName() == SqlTypeName.DOUBLE;
{code}
Since Hive might use other fractional types for the aggregation, we might end
up using the wrong type of aggregation in Druid. We could extend the check as
follows:
{code}
final boolean b =
SqlTypeName.FRACTIONAL_TYPES.contains(aggCall.getType().getSqlTypeName());
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CALCITE-1619) CAST is ignored by rules pushing operators into DruidQuery
Jesus Camacho Rodriguez created CALCITE-1619: Summary: CAST is ignored by rules pushing operators into DruidQuery Key: CALCITE-1619 URL: https://issues.apache.org/jira/browse/CALCITE-1619 Project: Calcite Issue Type: Bug Components: druid Affects Versions: 1.11.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.12.0 This could lead to incorrect semantics, for instance if we are casting a textual column (dimension) to a numerical value and then filtering. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CALCITE-1589) Druid adapter: timeseries query shows all days, even if no data
Jesus Camacho Rodriguez created CALCITE-1589:
Summary: Druid adapter: timeseries query shows all days, even if
no data
Key: CALCITE-1589
URL: https://issues.apache.org/jira/browse/CALCITE-1589
Project: Calcite
Issue Type: Bug
Components: druid
Reporter: Jesus Camacho Rodriguez
Assignee: Julian Hyde
Priority: Critical
Fix For: 1.12.0
Following query is transformed into timeseries Druid query which yields
different results in Calcite vs Druid, since it will show all values for the
given time granularity, even if there is no data for the given _i\_brand\_id_.
In Druid:
{code:sql}
SELECT floor_day(`__time`) as `granularity`, max(ss_quantity),
sum(ss_wholesale_cost)
FROM store_sales_sold_time_subset
WHERE i_brand_id = 10001009
GROUP BY floor_day(`__time`)
ORDER BY `granularity`;
OK
1999-11-01 00:00:00 45 37.47
1999-11-02 00:00:00 -92233720368547758080.0
1999-11-03 00:00:00 -92233720368547758080.0
1999-11-04 00:00:00 39 61.52
1999-11-05 00:00:00 74 145.84
1999-11-06 00:00:00 62 14.5
1999-11-07 00:00:00 -92233720368547758080.0
1999-11-08 00:00:00 5 34.08
1999-11-09 00:00:00 -92233720368547758080.0
1999-11-10 00:00:00 -92233720368547758080.0
1999-11-11 00:00:00 -92233720368547758080.0
1999-11-12 00:00:00 66 67.22
1999-11-13 00:00:00 -92233720368547758080.0
1999-11-14 00:00:00 -92233720368547758080.0
1999-11-15 00:00:00 -92233720368547758080.0
1999-11-16 00:00:00 60 96.37
1999-11-17 00:00:00 50 79.11
1999-11-18 00:00:00 -92233720368547758080.0
1999-11-19 00:00:00 -92233720368547758080.0
1999-11-20 00:00:00 -92233720368547758080.0
1999-11-21 00:00:00 -92233720368547758080.0
1999-11-22 00:00:00 -92233720368547758080.0
1999-11-23 00:00:00 57 17.69
1999-11-24 00:00:00 -92233720368547758080.0
1999-11-25 00:00:00 -92233720368547758080.0
1999-11-26 00:00:00 -92233720368547758080.0
1999-11-27 00:00:00 86 91.59
1999-11-28 00:00:00 -92233720368547758080.0
1999-11-29 00:00:00 93 136.48
1999-11-30 00:00:00 -92233720368547758080.0
{code:sql}
SELECT floor_day(`__time`) as `granularity`, max(ss_quantity),
sum(ss_wholesale_cost)
FROM store_sales_sold_time_subset_calcite
WHERE i_brand_id = 10001009
GROUP BY floor_day(`__time`)
ORDER BY `granularity`;
OK
1999-11-01 00:00:00 45 37.47
1999-11-04 00:00:00 39 61.52
1999-11-05 00:00:00 74 145.84
1999-11-06 00:00:00 62 14.5
1999-11-08 00:00:00 5 34.08
1999-11-12 00:00:00 66 67.22
1999-11-16 00:00:00 60 96.36
1999-11-17 00:00:00 50 79.11
1999-11-23 00:00:00 57 17.688
1999-11-27 00:00:00 86 91.59
1999-11-29 00:00:00 93 136.48
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1587) Druid adapter: topN returns approximate results
Jesus Camacho Rodriguez created CALCITE-1587: Summary: Druid adapter: topN returns approximate results Key: CALCITE-1587 URL: https://issues.apache.org/jira/browse/CALCITE-1587 Project: Calcite Issue Type: Bug Components: druid Affects Versions: 1.11.0 Reporter: Jesus Camacho Rodriguez Assignee: Julian Hyde Fix For: 1.12.0 Currently, we convert to _topN_ queries. However, metrics returned by Druid will be approximate values. Thus, probably we should not convert to Druid topN queries and rather always use Druid groupBy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1579) Druid adapter: wrong semantics of groupBy query limit with granularity
Jesus Camacho Rodriguez created CALCITE-1579:
Summary: Druid adapter: wrong semantics of groupBy query limit
with granularity
Key: CALCITE-1579
URL: https://issues.apache.org/jira/browse/CALCITE-1579
Project: Calcite
Issue Type: Bug
Components: druid
Affects Versions: 1.11.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Priority: Critical
Similar to CALCITE-1578, but for GroupBy queries. Limit is applied per
granularity unit, not globally for the query.
Currently, the following SQL query infers granularity 'day' for Druid _groupBy_
and pushes the limit, which is incorrect.
{code:sql}
SELECT i_brand_id, floor_day(`__time`), max(ss_quantity),
sum(ss_wholesale_cost) as s
FROM store_sales_sold_time_subset
GROUP BY i_brand_id, floor_day(`__time`)
ORDER BY s
LIMIT 10;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1577) Druid adapter: Incorrect result - limit on timestamp disappears
Jesus Camacho Rodriguez created CALCITE-1577:
Summary: Druid adapter: Incorrect result - limit on timestamp
disappears
Key: CALCITE-1577
URL: https://issues.apache.org/jira/browse/CALCITE-1577
Project: Calcite
Issue Type: Bug
Components: druid
Affects Versions: 1.11.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Priority: Critical
This can be observed with the following query:
{code:sql}
SELECT DISTINCT `__time`
FROM store_sales_sold_time_subset
ORDER BY `__time` ASC
LIMIT 10;
{code}
Query is translated to Druid _timeseries_, but _limit_ operator disappears.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1421) Update website documentation to verify a release
Jesus Camacho Rodriguez created CALCITE-1421: Summary: Update website documentation to verify a release Key: CALCITE-1421 URL: https://issues.apache.org/jira/browse/CALCITE-1421 Project: Calcite Issue Type: Bug Components: site Affects Versions: 1.9.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Fix For: 1.11.0 Now we replace digest files with a single digest; thus, we need to update the documentation for the way we validate the signature for the artifacts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1420) Allow Calcite JDBC Driver minor version to be greater than 9
Jesus Camacho Rodriguez created CALCITE-1420:
Summary: Allow Calcite JDBC Driver minor version to be greater
than 9
Key: CALCITE-1420
URL: https://issues.apache.org/jira/browse/CALCITE-1420
Project: Calcite
Issue Type: Bug
Affects Versions: 1.9.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Priority: Minor
Fix For: 1.10.0
l1031 in /core/src/test/java/org/apache/calcite/test/JdbcTest.java
{code}
assertTrue(driverMinor >= 0 && driverMinor < 10);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1413) New CASE statement simplification
Jesus Camacho Rodriguez created CALCITE-1413:
Summary: New CASE statement simplification
Key: CALCITE-1413
URL: https://issues.apache.org/jira/browse/CALCITE-1413
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.9.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Another possible simplification for CASE statements.
{code:sql}
CASE
WHEN p1 THEN x
WHEN p2 THEN y
ELSE TRUE
END
{code}
can be rewritten into:
{code:sql}
(p1 and x) or (p2 and y and not(p1)) or (not(p1) and not(p2))
{code}
if p1...pn cannot be nullable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1402) Druid Filter translation incorrect if input reference is in RHS of comparison
Jesus Camacho Rodriguez created CALCITE-1402: Summary: Druid Filter translation incorrect if input reference is in RHS of comparison Key: CALCITE-1402 URL: https://issues.apache.org/jira/browse/CALCITE-1402 Project: Calcite Issue Type: Bug Components: druid Affects Versions: 1.9.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez It should have been fixed as part of CALCITE-1357/CALCITE-1358. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1400) AggregatePullUpConstantsRule might adjust aggregation function parameters indices wrongly
Jesus Camacho Rodriguez created CALCITE-1400: Summary: AggregatePullUpConstantsRule might adjust aggregation function parameters indices wrongly Key: CALCITE-1400 URL: https://issues.apache.org/jira/browse/CALCITE-1400 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.9.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez I explored a bit and I think this is a piece missing in CALCITE-1038. Introducing a Project operator below as part of the rewriting was removed, and thus, there is no need to adjust the indices of the aggregation function parameters. However, the code is still there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1398) Change visibility of RelFieldTrimmer utility methods
Jesus Camacho Rodriguez created CALCITE-1398: Summary: Change visibility of RelFieldTrimmer utility methods Key: CALCITE-1398 URL: https://issues.apache.org/jira/browse/CALCITE-1398 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.9.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez The goal is that its subclasses can use them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1397) ClassCastException in FilterReduceExpressionsRule
Jesus Camacho Rodriguez created CALCITE-1397:
Summary: ClassCastException in FilterReduceExpressionsRule
Key: CALCITE-1397
URL: https://issues.apache.org/jira/browse/CALCITE-1397
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.9.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
L184 in ReduceExpressionsRule.
{code}
...
if (newConditionExp instanceof RexCall) {
RexCall rexCall = (RexCall) newConditionExp;
boolean reverse = rexCall.getKind() == SqlKind.NOT;
if (reverse) {
rexCall = (RexCall) rexCall.getOperands().get(0);
}
reduceNotNullableFilter(call, filter, rexCall, reverse);
}
...
{code}
When we take the _NOT_ input, we do not consider that it might not be a
RexCall, which may lead to the ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1377) JdbcTest.testUnicode failure
Jesus Camacho Rodriguez created CALCITE-1377:
Summary: JdbcTest.testUnicode failure
Key: CALCITE-1377
URL: https://issues.apache.org/jira/browse/CALCITE-1377
Project: Calcite
Issue Type: Test
Affects Versions: 1.9.0
Reporter: Jesus Camacho Rodriguez
Assignee: Julian Hyde
Test suite fails when running:
{{mvn clean && mvn -Pit install}}
{code}
Results :
Failed tests:
testUnicode(org.apache.calcite.test.JdbcTest): java.sql.SQLException: Error
while executing SQL "select * from "employee" where "full_name" = '英国'": From
line 1, column 15 to line 1, column 24: Table 'employee' not found
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:147)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:208)
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:481)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.throws_(CalciteAssert.java:1242)
at org.apache.calcite.test.JdbcTest.testUnicode(JdbcTest.java:5764)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:35)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:115)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
at com.sun.proxy.$Proxy0.invoke(Unknown Source)
at
org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:150)
at
org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:91)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1,
column 15 to line 1, column 24: Table 'employee' not found
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:405)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:799)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:784)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4044)
at
org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:106)
at
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:86)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:865)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:846)
[jira] [Created] (CALCITE-1376) Add test that sorts all operators based on precedence and checks their order
Jesus Camacho Rodriguez created CALCITE-1376:
Summary: Add test that sorts all operators based on precedence and
checks their order
Key: CALCITE-1376
URL: https://issues.apache.org/jira/browse/CALCITE-1376
Project: Calcite
Issue Type: Test
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Julian Hyde
Priority: Minor
Per discussion in CALCITE-1095, [~julianhyde] wrote:
{quote}
I think we should have a test that sorts all operators based on precedence and
checks that they come out in the order that we expect, per
https://github.com/julianhyde/calcite/blob/1095-not-precedence/site/_docs/reference.md#operator-precedence.
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1366) Metadata provider for predicates pulls up expressions without references incorrectly
Jesus Camacho Rodriguez created CALCITE-1366:
Summary: Metadata provider for predicates pulls up expressions
without references incorrectly
Key: CALCITE-1366
URL: https://issues.apache.org/jira/browse/CALCITE-1366
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.9.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
{code}
...
for (RexNode r : inputInfo.pulledUpPredicates) {
ImmutableBitSet rCols = RelOptUtil.InputFinder.bits(r);
if (groupKeys.contains(rCols)) {
r = r.accept(new RexPermuteInputsShuttle(m, input));
aggPullUpPredicates.add(r);
}
}
...
{code}
The check does not take into account that _rCols_ might be empty, and then _r_
cannot be pulled up e.g. count(*).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1365) Introduce HiveUnionPullUpConstantsRule
Jesus Camacho Rodriguez created CALCITE-1365: Summary: Introduce HiveUnionPullUpConstantsRule Key: CALCITE-1365 URL: https://issues.apache.org/jira/browse/CALCITE-1365 Project: Calcite Issue Type: New Feature Components: core Affects Versions: 1.9.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Create a rule that pulls up constants through a Union operator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1358) Push filters on time dimension to Druid
Jesus Camacho Rodriguez created CALCITE-1358: Summary: Push filters on time dimension to Druid Key: CALCITE-1358 URL: https://issues.apache.org/jira/browse/CALCITE-1358 Project: Calcite Issue Type: Bug Components: druid Affects Versions: 1.9.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Porting work done in HIVE-14217. Logic should split the filter conditions into two parts: those predicates referred to the time dimension, and predicates referred to other columns. Then, the predicates on the time dimension should be translated into Druid intervals, possibly consolidating those ranges e.g. to detect overlapping. The other predicates will go into the Druid filter field. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1357) Recognize Druid Timeseries and TopN queries in DruidQuery
Jesus Camacho Rodriguez created CALCITE-1357: Summary: Recognize Druid Timeseries and TopN queries in DruidQuery Key: CALCITE-1357 URL: https://issues.apache.org/jira/browse/CALCITE-1357 Project: Calcite Issue Type: Bug Affects Versions: 1.9.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Port the work done in HIVE-14217 to recognize Timeseries and TopN queries to Calcite. This includes a rule to push SortLimit into DruidQuery, which can lead to creating TopN queries. This rule can help to push sorting and limit into GroupBy queries, and limit to Select queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1344) Incorrect inferred precision when BigDecimal value is less than 1
Jesus Camacho Rodriguez created CALCITE-1344:
Summary: Incorrect inferred precision when BigDecimal value is
less than 1
Key: CALCITE-1344
URL: https://issues.apache.org/jira/browse/CALCITE-1344
Project: Calcite
Issue Type: Bug
Affects Versions: 1.8.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Method {{makeExactLiteral(BigDecimal)}} in RexBuilder infers the incorrect
precision when decimal < 1 e.g. for 0.06 it infers the type to be Decimal(1,2)
instead of Decimal(3,2).
{code:java}
/**
* Creates a numeric literal.
*/
public RexLiteral makeExactLiteral(BigDecimal bd) {
RelDataType relType;
int scale = bd.scale();
long l = bd.unscaledValue().longValue();
assert scale >= 0;
assert scale <= typeFactory.getTypeSystem().getMaxNumericScale() : scale;
assert BigDecimal.valueOf(l, scale).equals(bd);
if (scale == 0) {
if ((l >= Integer.MIN_VALUE) && (l <= Integer.MAX_VALUE)) {
relType = typeFactory.createSqlType(SqlTypeName.INTEGER);
} else {
relType = typeFactory.createSqlType(SqlTypeName.BIGINT);
}
} else {
int precision = bd.unscaledValue().abs().toString().length();
relType =
typeFactory.createSqlType(SqlTypeName.DECIMAL, precision, scale);
}
return makeExactLiteral(bd, relType);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CALCITE-1220) Extend simplify for reducing expressions II
Jesus Camacho Rodriguez created CALCITE-1220: Summary: Extend simplify for reducing expressions II Key: CALCITE-1220 URL: https://issues.apache.org/jira/browse/CALCITE-1220 Project: Calcite Issue Type: Bug Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez We would extend expression simplification, e.g.: - add walker to perform recursive simplification of expressions for Filter operators, and - extends simplification for CASE expressions to cover more cases e.g. if all return values are equal (including ELSE), CASE can be removed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CALCITE-1199) Incorrect trimming of CHAR when performing cast to VARCHAR
Jesus Camacho Rodriguez created CALCITE-1199:
Summary: Incorrect trimming of CHAR when performing cast to VARCHAR
Key: CALCITE-1199
URL: https://issues.apache.org/jira/browse/CALCITE-1199
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Assume the value ' ' with type CHAR(1) that needs to be cast to VARCHAR(10).
The method {{makeCast}} (line 488 in RexBuilder.java) trims the right hand side
of the CHAR value, but this seems incorrect as it could lead to incorrect
results.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
