[jira] [Created] (CALCITE-4722) CHECKERFRAMEWORK job always fails in Travis CI
Stamatis Zampetakis created CALCITE-4722: Summary: CHECKERFRAMEWORK job always fails in Travis CI Key: CALCITE-4722 URL: https://issues.apache.org/jira/browse/CALCITE-4722 Project: Calcite Issue Type: Bug Reporter: Stamatis Zampetakis A sample failing job can be seen [here|https://app.travis-ci.com/github/apache/calcite/jobs/529699168]. The stacktrace is shown below: {noformat} Build calcite FAILURE reason: Caused by: java.lang.NoClassDefFoundError: org/jetbrains/kotlin/cli/common/PropertiesKt at org.jetbrains.kotlin.gradle.plugin.KotlinBasePluginWrapper.apply(KotlinPluginWrapper.kt:64) at org.jetbrains.kotlin.gradle.plugin.KotlinBasePluginWrapper.apply(KotlinPluginWrapper.kt:41) at org.gradle.api.internal.plugins.ImperativeOnlyPluginTarget.applyImperative(ImperativeOnlyPluginTarget.java:43) at org.gradle.api.internal.plugins.RuleBasedPluginTarget.applyImperative(RuleBasedPluginTarget.java:51) at org.gradle.api.internal.plugins.DefaultPluginManager.addPlugin(DefaultPluginManager.java:177) at org.gradle.api.internal.plugins.DefaultPluginManager.access$100(DefaultPluginManager.java:51) at org.gradle.api.internal.plugins.DefaultPluginManager$AddPluginBuildOperation.run(DefaultPluginManager.java:272) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26) at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:75) at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:68) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:153) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:68) at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:56) at org.gradle.internal.operations.DefaultBuildOperationExecutor.lambda$run$1(DefaultBuildOperationExecutor.java:71) at org.gradle.internal.operations.UnmanagedBuildOperationWrapper.runWithUnmanagedSupport(UnmanagedBuildOperationWrapper.java:45) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:71) at org.gradle.api.internal.plugins.DefaultPluginManager.lambda$doApply$0(DefaultPluginManager.java:157) at org.gradle.configuration.internal.DefaultUserCodeApplicationContext.apply(DefaultUserCodeApplicationContext.java:43) at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:156) at org.gradle.api.internal.plugins.DefaultPluginManager.apply(DefaultPluginManager.java:136) at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.lambda$applyLegacyPlugins$2(DefaultPluginRequestApplicator.java:138) at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyPlugin(DefaultPluginRequestApplicator.java:176) at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.lambda$applyLegacyPlugins$3(DefaultPluginRequestApplicator.java:136) at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyLegacyPlugins(DefaultPluginRequestApplicator.java:136) at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyPlugins(DefaultPluginRequestApplicator.java:120) at org.gradle.kotlin.dsl.provider.PluginRequestsHandler.handle(PluginRequestsHandler.kt:48) at org.gradle.kotlin.dsl.provider.StandardKotlinScriptEvaluator$InterpreterHost.applyPluginsTo(KotlinScriptEvaluator.kt:203) at org.gradle.kotlin.dsl.execution.Interpreter$ProgramHost.applyPluginsTo(Interpreter.kt:369) at org.gradle.kotlin.dsl.execution.Interpreter$ProgramHost.eval(Interpreter.kt:501) at org.gradle.kotlin.dsl.execution.Interpreter.eval(Interpreter.kt:199) at org.gradle.kotlin.dsl.provider.StandardKotlinScriptEvaluator.evaluate(KotlinScriptEvaluator.kt:124) at org.gradle.kotlin.dsl.provider.KotlinScriptPluginFactory$create$1.invoke(KotlinScriptPluginFactory.kt:51) at org.gradle.kotlin.dsl.provider.KotlinScriptPluginFactory$create$1.invoke(KotlinScriptPluginFactory.kt:36) at org.gradle.kotlin.dsl.provider.KotlinScriptPlugin.apply(KotlinScriptPlugin.kt:34) at org.gradle.configuration.BuildOperationScriptPlugin$1.run(BuildOperationScriptPlugin.java:65) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29) at org.gradle.internal.operations.DefaultBuild
Re: Review request: CALCITE-4652 (fix AggregateExpandDistinctAggregatesRule when SUM type is expanded)
Hi Calcite Devs. I just remind about review/merge the patch for the issue CALCITE-4652 [1], see PR#2439 [2]. I've fixed the patch according with Julian comments. Also PR contains two 'LGTM' comments. Is the patch ready for merge? [1]. https://issues.apache.org/jira/browse/CALCITE-4652 [2]. https://github.com/apache/calcite/pull/2439 On 12.07.2021 15:05, xiong duan wrote: Hi. Ledkov. I'll do some code reviews in the next two days. Taras Ledkov 于2021年7月12日周一 下午7:58写道: Hi, Please review the patch for the issue CALCITE-4652 [1], see PR#2439 [2]. I tried to draw attention to the issue in the topic: "[HELP] Return type of the SUM aggregate function and AggregateExpandDistinctAggregatesRule", but did not receive any answer, so I do not give a link to the discussion. Stamatis advised me to send a reminder to the devlist. [1]. https://issues.apache.org/jira/browse/CALCITE-4652 [2]. https://github.com/apache/calcite/pull/2439 -- Taras Ledkov Mail-To: tled...@gridgain.com -- Taras Ledkov Mail-To: tled...@gridgain.com
Materialized View and Lattice Suggester help
Hi Calcite devs, Today our team deploys a Calcite service that uses Daily OLAP Cube tables and fine grain Fact tables. Daily OLAP Cube tables are used to vastly improve latency when querying for a wide date range but comes with a restriction – the date range *must* be UTC day granularity. When a request comes into the Cube endpoint, our service picks the best Cube, creates a Logical Plan for that query and lets Calcite optimize it into Enumerable. Our Service has matured to the point where we would like to *remove* the UTC day granularity restriction and would like to do Query Rewriting and I am writing to you all to figure out the best way to do that. Calcite supports Materialized Views and Lattices and I haven't been able to pull this together. Here is a motivating example: ┌ │ SELECT "SIM_ID", SUM("BYTES_DOWNLINK") + SUM(BYTES_UPLINK) as "SUM_BYTES" │ FROM "kudu"."WirelessEvents" │ WHERE "ACCOUNT_ID" = 18789 AND ("START_TIME" >= TIMESTAMP '2021-07-31 11:30:01' AND "START_TIME" < TIMESTAMP '2021-09-01 03:00:00') │ GROUP BY "SIM_ID" │ ORDER BY "SIM_ID" DESC; └ This is a query we would like to rewrite to have it use the UTC Daily OLAP Cube. One possible rewrite of this: ┌ │ SELECT "SIM_ID", SUM("SUM_BYTES_DOWNLINK") + SUM("SUM_BYTES_UPLINK") AS "SUM_BYTES" │ │ FROM ( │ /* Cube Table query with all the filters on dimensions */ │ /* First UTC Day and last UTC Day */ │ SELECT "SIM_ID", "SUM_BYTES_DOWNLINK", SUM_BYTES_UPLINK │ FROM "kudu"."Wireless-Daily-Aggregation" │ WHERE "ACCOUNT_ID" = 18789 AND ("START_TIME" >= TIMESTAMP '2021-08-01 00:00:00' AND "START_TIME" < TIMESTAMP '2021-08-31 23:59:59') │ │ UNION ALL │ │ /* Fact Table query with all the filters on dimensions */ │ /* Aliasing measure fields into their Cube table values */ │ /* Date filter is from startDate to the first UTC day in query */ │ SELECT "SIM_ID", "BYTES_DOWNLINK" AS "SUM_BYTES_DOWNLINK", "BYTES_UPLINK" AS "SUM_BYTES_UPLINK" │ FROM "kudu"."WirelessEvents" │ WHERE "ACCOUNT_ID" = 18789 AND ("START_TIME" >= TIMESTAMP '2021-07-31 11:30:01' AND "START_TIME" < TIMESTAMP '2021-08-01 00:00:00') │ │ UNION ALL │ │ /* Fact Table query with all the filters on dimensions */ │ /* Aliasing measure fields into their Cube table values */ │ /* Date filter is from last UTC day to endDate of the query */ │ SELECT "SIM_ID", "BYTES_DOWNLINK" AS "SUM_BYTES_DOWNLINK", "BYTES_UPLINK" AS "SUM_BYTES_UPLINK" │ FROM "kudu"."WirelessEvents" │ WHERE "ACCOUNT_ID" = 18789 AND ("START_TIME" >= TIMESTAMP '2021-08-31 23:59:59' AND "START_TIME" < TIMESTAMP '2021-09-01 03:00:00') │ ) │ │ GROUP BY "SIM_ID" │ ORDER BY "SIM_ID" DESC; └ In this instance, Materialized View rule will need to: 1. Create two scans (or one with a Disjunction) on the ranges outside of UTC boundary 2. Create `Projection' that changes the names of the Fact columns to match the measure names – `BYTES_UPLINK' becomes `SUM_BYTES_UPLINK' I am overwhelmed by [Materialized View] rules and [Lattice Suggester] and hoping you all could describe in broad terms how you would approach this task. What do I need to configure? What do I need to build? Thanks so much ! [Materialized View] https://calcite.apache.org/docs/materialized_views.html [Lattice Suggester] https://calcite.apache.org/docs/lattice.html
[jira] [Created] (CALCITE-4723) Check whether JDBC adapter generates "GROUP BY ()" against Oracle, DB2, MSSQL
Julian Hyde created CALCITE-4723: Summary: Check whether JDBC adapter generates "GROUP BY ()" against Oracle, DB2, MSSQL Key: CALCITE-4723 URL: https://issues.apache.org/jira/browse/CALCITE-4723 Project: Calcite Issue Type: Bug Reporter: Julian Hyde Oracle, DB2 and MSSQL have non-standard semantics for "GROUP BY ()". Standard behavior is to always return one "grand total" row, but [Oracle, DB2 and MSSQL return no rows if the input is empty|https://blog.jooq.org/2018/05/25/how-to-group-by-nothing-in-sql/]. Calcite's semantics is that "GROUP BY ()" always returns one row, and the JDBC adapter currently assumes that all back ends have the same semantics. On back ends that have different semantics, some queries might be giving incorrect results. I suggest the following remedy: * Add a {{SqlDialect}} method {{boolean omitGrandTotalOnEmptyInput()}} * Run the test suite, and see whether we ever generate "GROUP BY ()" on one of the affected dialects. Try to write a test case where we do this. * Modify the dialects to generate safe SQL in these cases (possibly "GROUP BY ()", or possibly something else). As the above article notes, it is particularly difficult to find SQL that works for MSSQL, because it bumps into the no-constants rule (see CALCITE-4702) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4724) ClickHouseSqlDialect `supportsAliasedValues` should return false.
Enze Liu created CALCITE-4724: - Summary: ClickHouseSqlDialect `supportsAliasedValues` should return false. Key: CALCITE-4724 URL: https://issues.apache.org/jira/browse/CALCITE-4724 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.27.0 Reporter: Enze Liu We use ClickHouseSqlDialect to do some sql optimize. for sql `select 1`, in the process of sqlNode -> relNode -> sqlNode, the sql string will be transformed to {code:java} SELECT * FROM (VALUES (1)) AS `t` (`EXPR$0`) {code} Since clickhouse is not support AliasedValues, all we need to do is to extend ClickHouseSqlDialect and make `supportsAliasedValues` return false. Maybe this kind of behavior can integrated into core? If needed, the pull request is ready. -- This message was sent by Atlassian Jira (v8.3.4#803005)