[jira] [Created] (CALCITE-4722) CHECKERFRAMEWORK job always fails in Travis CI
Stamatis Zampetakis created CALCITE-4722:
Summary: CHECKERFRAMEWORK job always fails in Travis CI
Key: CALCITE-4722
URL: https://issues.apache.org/jira/browse/CALCITE-4722
Project: Calcite
Issue Type: Bug
Reporter: Stamatis Zampetakis
A sample failing job can be seen
[here|https://app.travis-ci.com/github/apache/calcite/jobs/529699168].
The stacktrace is shown below:
{noformat}
Build calcite FAILURE reason:
Caused by: java.lang.NoClassDefFoundError:
org/jetbrains/kotlin/cli/common/PropertiesKt
at
org.jetbrains.kotlin.gradle.plugin.KotlinBasePluginWrapper.apply(KotlinPluginWrapper.kt:64)
at
org.jetbrains.kotlin.gradle.plugin.KotlinBasePluginWrapper.apply(KotlinPluginWrapper.kt:41)
at
org.gradle.api.internal.plugins.ImperativeOnlyPluginTarget.applyImperative(ImperativeOnlyPluginTarget.java:43)
at
org.gradle.api.internal.plugins.RuleBasedPluginTarget.applyImperative(RuleBasedPluginTarget.java:51)
at
org.gradle.api.internal.plugins.DefaultPluginManager.addPlugin(DefaultPluginManager.java:177)
at
org.gradle.api.internal.plugins.DefaultPluginManager.access$100(DefaultPluginManager.java:51)
at
org.gradle.api.internal.plugins.DefaultPluginManager$AddPluginBuildOperation.run(DefaultPluginManager.java:272)
at
org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
at
org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
at
org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:75)
at
org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:68)
at
org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:153)
at
org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:68)
at
org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:56)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor.lambda$run$1(DefaultBuildOperationExecutor.java:71)
at
org.gradle.internal.operations.UnmanagedBuildOperationWrapper.runWithUnmanagedSupport(UnmanagedBuildOperationWrapper.java:45)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:71)
at
org.gradle.api.internal.plugins.DefaultPluginManager.lambda$doApply$0(DefaultPluginManager.java:157)
at
org.gradle.configuration.internal.DefaultUserCodeApplicationContext.apply(DefaultUserCodeApplicationContext.java:43)
at
org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:156)
at
org.gradle.api.internal.plugins.DefaultPluginManager.apply(DefaultPluginManager.java:136)
at
org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.lambda$applyLegacyPlugins$2(DefaultPluginRequestApplicator.java:138)
at
org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyPlugin(DefaultPluginRequestApplicator.java:176)
at
org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.lambda$applyLegacyPlugins$3(DefaultPluginRequestApplicator.java:136)
at
org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyLegacyPlugins(DefaultPluginRequestApplicator.java:136)
at
org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyPlugins(DefaultPluginRequestApplicator.java:120)
at
org.gradle.kotlin.dsl.provider.PluginRequestsHandler.handle(PluginRequestsHandler.kt:48)
at
org.gradle.kotlin.dsl.provider.StandardKotlinScriptEvaluator$InterpreterHost.applyPluginsTo(KotlinScriptEvaluator.kt:203)
at
org.gradle.kotlin.dsl.execution.Interpreter$ProgramHost.applyPluginsTo(Interpreter.kt:369)
at
org.gradle.kotlin.dsl.execution.Interpreter$ProgramHost.eval(Interpreter.kt:501)
at org.gradle.kotlin.dsl.execution.Interpreter.eval(Interpreter.kt:199)
at
org.gradle.kotlin.dsl.provider.StandardKotlinScriptEvaluator.evaluate(KotlinScriptEvaluator.kt:124)
at
org.gradle.kotlin.dsl.provider.KotlinScriptPluginFactory$create$1.invoke(KotlinScriptPluginFactory.kt:51)
at
org.gradle.kotlin.dsl.provider.KotlinScriptPluginFactory$create$1.invoke(KotlinScriptPluginFactory.kt:36)
at
org.gradle.kotlin.dsl.provider.KotlinScriptPlugin.apply(KotlinScriptPlugin.kt:34)
at
org.gradle.configuration.BuildOperationScriptPlugin$1.run(BuildOperationScriptPlugin.java:65)
at
org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
at
org.gradle.internal.operations.DefaultBuild
Re: Review request: CALCITE-4652 (fix AggregateExpandDistinctAggregatesRule when SUM type is expanded)
Hi Calcite Devs. I just remind about review/merge the patch for the issue CALCITE-4652 [1], see PR#2439 [2]. I've fixed the patch according with Julian comments. Also PR contains two 'LGTM' comments. Is the patch ready for merge? [1]. https://issues.apache.org/jira/browse/CALCITE-4652 [2]. https://github.com/apache/calcite/pull/2439 On 12.07.2021 15:05, xiong duan wrote: Hi. Ledkov. I'll do some code reviews in the next two days. Taras Ledkov 于2021年7月12日周一 下午7:58写道： Hi, Please review the patch for the issue CALCITE-4652 [1], see PR#2439 [2]. I tried to draw attention to the issue in the topic: "[HELP] Return type of the SUM aggregate function and AggregateExpandDistinctAggregatesRule", but did not receive any answer, so I do not give a link to the discussion. Stamatis advised me to send a reminder to the devlist. [1]. https://issues.apache.org/jira/browse/CALCITE-4652 [2]. https://github.com/apache/calcite/pull/2439 -- Taras Ledkov Mail-To: tled...@gridgain.com -- Taras Ledkov Mail-To: tled...@gridgain.com
Materialized View and Lattice Suggester help
Hi Calcite devs,
Today our team deploys a Calcite service that uses Daily OLAP Cube
tables and fine grain Fact tables. Daily OLAP Cube tables are used to
vastly improve latency when querying for a wide date range but comes
with a restriction – the date range *must* be UTC day granularity. When
a request comes into the Cube endpoint, our service picks the best Cube,
creates a Logical Plan for that query and lets Calcite optimize it into
Enumerable. Our Service has matured to the point where we would like to
*remove* the UTC day granularity restriction and would like to do Query
Rewriting and I am writing to you all to figure out the best way to do
that. Calcite supports Materialized Views and Lattices and I haven't
been able to pull this together. Here is a motivating example:
┌
│ SELECT "SIM_ID", SUM("BYTES_DOWNLINK") + SUM(BYTES_UPLINK) as "SUM_BYTES"
│ FROM "kudu"."WirelessEvents"
│ WHERE "ACCOUNT_ID" = 18789 AND ("START_TIME" >= TIMESTAMP '2021-07-31
11:30:01' AND "START_TIME" < TIMESTAMP '2021-09-01 03:00:00')
│ GROUP BY "SIM_ID"
│ ORDER BY "SIM_ID" DESC;
└
This is a query we would like to rewrite to have it use the UTC Daily
OLAP Cube. One possible rewrite of this:
┌
│ SELECT "SIM_ID", SUM("SUM_BYTES_DOWNLINK") + SUM("SUM_BYTES_UPLINK") AS
"SUM_BYTES"
│
│ FROM (
│ /* Cube Table query with all the filters on dimensions */
│ /* First UTC Day and last UTC Day */
│ SELECT "SIM_ID", "SUM_BYTES_DOWNLINK", SUM_BYTES_UPLINK
│ FROM "kudu"."Wireless-Daily-Aggregation"
│ WHERE "ACCOUNT_ID" = 18789 AND ("START_TIME" >= TIMESTAMP '2021-08-01
00:00:00' AND "START_TIME" < TIMESTAMP '2021-08-31 23:59:59')
│
│ UNION ALL
│
│ /* Fact Table query with all the filters on dimensions */
│ /* Aliasing measure fields into their Cube table values */
│ /* Date filter is from startDate to the first UTC day in query */
│ SELECT "SIM_ID", "BYTES_DOWNLINK" AS "SUM_BYTES_DOWNLINK", "BYTES_UPLINK"
AS "SUM_BYTES_UPLINK"
│ FROM "kudu"."WirelessEvents"
│ WHERE "ACCOUNT_ID" = 18789 AND ("START_TIME" >= TIMESTAMP '2021-07-31
11:30:01' AND "START_TIME" < TIMESTAMP '2021-08-01 00:00:00')
│
│ UNION ALL
│
│ /* Fact Table query with all the filters on dimensions */
│ /* Aliasing measure fields into their Cube table values */
│ /* Date filter is from last UTC day to endDate of the query */
│ SELECT "SIM_ID", "BYTES_DOWNLINK" AS "SUM_BYTES_DOWNLINK", "BYTES_UPLINK"
AS "SUM_BYTES_UPLINK"
│ FROM "kudu"."WirelessEvents"
│ WHERE "ACCOUNT_ID" = 18789 AND ("START_TIME" >= TIMESTAMP '2021-08-31
23:59:59' AND "START_TIME" < TIMESTAMP '2021-09-01 03:00:00')
│ )
│
│ GROUP BY "SIM_ID"
│ ORDER BY "SIM_ID" DESC;
└
In this instance, Materialized View rule will need to:
1. Create two scans (or one with a Disjunction) on the ranges outside of
UTC boundary
2. Create `Projection' that changes the names of the Fact columns to
match the measure names – `BYTES_UPLINK' becomes `SUM_BYTES_UPLINK'
I am overwhelmed by [Materialized View] rules and [Lattice Suggester] and
hoping you all could describe in broad terms how you would approach this
task. What do I need to configure? What do I need to build?
Thanks so much !
[Materialized View]
https://calcite.apache.org/docs/materialized_views.html
[Lattice Suggester] https://calcite.apache.org/docs/lattice.html
[jira] [Created] (CALCITE-4723) Check whether JDBC adapter generates "GROUP BY ()" against Oracle, DB2, MSSQL
Julian Hyde created CALCITE-4723:
Summary: Check whether JDBC adapter generates "GROUP BY ()"
against Oracle, DB2, MSSQL
Key: CALCITE-4723
URL: https://issues.apache.org/jira/browse/CALCITE-4723
Project: Calcite
Issue Type: Bug
Reporter: Julian Hyde
Oracle, DB2 and MSSQL have non-standard semantics for "GROUP BY ()". Standard
behavior is to always return one "grand total" row, but [Oracle, DB2 and MSSQL
return no rows if the input is
empty|https://blog.jooq.org/2018/05/25/how-to-group-by-nothing-in-sql/].
Calcite's semantics is that "GROUP BY ()" always returns one row, and the JDBC
adapter currently assumes that all back ends have the same semantics. On back
ends that have different semantics, some queries might be giving incorrect
results.
I suggest the following remedy:
* Add a {{SqlDialect}} method {{boolean omitGrandTotalOnEmptyInput()}}
* Run the test suite, and see whether we ever generate "GROUP BY ()" on one of
the affected dialects. Try to write a test case where we do this.
* Modify the dialects to generate safe SQL in these cases (possibly "GROUP BY
()", or possibly something else). As the above article notes, it is
particularly difficult to find SQL that works for MSSQL, because it bumps into
the no-constants rule (see CALCITE-4702)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4724) ClickHouseSqlDialect `supportsAliasedValues` should return false.
Enze Liu created CALCITE-4724:
-
Summary: ClickHouseSqlDialect `supportsAliasedValues` should
return false.
Key: CALCITE-4724
URL: https://issues.apache.org/jira/browse/CALCITE-4724
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.27.0
Reporter: Enze Liu
We use ClickHouseSqlDialect to do some sql optimize.
for sql `select 1`, in the process of sqlNode -> relNode -> sqlNode, the sql
string will be transformed to
{code:java}
SELECT *
FROM (VALUES (1)) AS `t` (`EXPR$0`)
{code}
Since clickhouse is not support AliasedValues, all we need to do is to extend
ClickHouseSqlDialect and make `supportsAliasedValues` return false.
Maybe this kind of behavior can integrated into core?
If needed, the pull request is ready.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
