Dynamic tables inside a schema
Hi,
I'm creating a custom adapter by
1. Implementing org.apache.calcite.schema.SchemaFactory
2. Create schema that directly implments org.apache.calcite.schema.Schema
3. Inside the schema, tables that extends
org.apache.calcite.schema.impl.AbstractTable
Inside my schema implemenation, i'm overriding the below methods
import org.apache.calcite.schema.Table;
private Map tableMap;
@Override
protected Map getTableMap() {
if (tableMap == null) {
tableMap = createTableMap();
}
return tableMap;
}
@Override
public Table getTable(String name) {
return new AzStorageTable(this,name);
}
This adapter will be querying set of files stored in a location. And i don't
want to pre-populate all the file paths in the getTableMap() method. Is there a
way to add filepath to this during the query time dynamically ?
Overriding getTable(String name) has no effect because the validation layer
takes the list of table names form tableMap.
Few articles on the internet mention that Apache drill does something similar
with hdfs paths. But I'm unable to understand how Apache Drill is able to
achieve it.
I need help in achieving this. Thanks for the help/suggestions in advance.
Thanks
Arun
Re: Requesting a review on a PR----[CALCITE-4661]
I will take a look within this week. -Rui On Wed, Jun 30, 2021 at 6:55 PM xiong duan wrote: > Hi Follower, Please review this PR, So that I can make some improvements. > Thank you. >
[jira] [Created] (CALCITE-4675) Error executing query with SUM and multiplication via JdbcAdapter
Stamatis Zampetakis created CALCITE-4675:
Summary: Error executing query with SUM and multiplication via
JdbcAdapter
Key: CALCITE-4675
URL: https://issues.apache.org/jira/browse/CALCITE-4675
Project: Calcite
Issue Type: Bug
Affects Versions: 1.27.0
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
Fix For: 1.28.0
{code:sql}
SELECT s."customer_id", sum(s."store_sales" * s."store_cost")
FROM "sales_fact_1997" AS s
JOIN "customer" AS c ON s."customer_id" = c."customer_id"
GROUP BY s."customer_id"
{code}
The query above using a custom ruleset and the standard JdbcRules generates the
following SQL (e.g., RedShift dialect):
{code:sql}
SELECT "t"."customer_id", SUM("t"."*")
FROM
(SELECT "customer_id", "store_sales" * "store_cost" AS "*"
FROM "foodmart"."sales_fact_1997") AS "t"
INNER JOIN
(SELECT "customer_id"
FROM "foodmart"."customer") AS "t0" ON "t"."customer_id" = "t0"."customer_id"
GROUP BY "t"."customer_id"
{code}
and raises exceptions when executed over some DBMS which cannot handle STAR
symbol (\*) properly.
In RedShift, the query above will raise the following error:
{noformat}
Caused by: com.amazon.redshift.util.RedshiftException: ERROR: function
sum(record) does not exist
Hint: No function matches the given name and argument types. You may need to
add explicit type casts.
at
com.amazon.redshift.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2579)
at
com.amazon.redshift.core.v3.QueryExecutorImpl.processResultsOnThread(QueryExecutorImpl.java:2253)
at
com.amazon.redshift.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1880)
at
com.amazon.redshift.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1872)
at
com.amazon.redshift.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:368)
at
com.amazon.redshift.jdbc.RedshiftStatementImpl.executeInternal(RedshiftStatementImpl.java:514)
at
com.amazon.redshift.jdbc.RedshiftStatementImpl.execute(RedshiftStatementImpl.java:435)
at
com.amazon.redshift.jdbc.RedshiftPreparedStatement.executeWithFlags(RedshiftPreparedStatement.java:200)
at
com.amazon.redshift.jdbc.RedshiftPreparedStatement.executeQuery(RedshiftPreparedStatement.java:115)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4674) Excess quotes in generated SQL when STAR is a column alias
Stamatis Zampetakis created CALCITE-4674:
Summary: Excess quotes in generated SQL when STAR is a column alias
Key: CALCITE-4674
URL: https://issues.apache.org/jira/browse/CALCITE-4674
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.27.0
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
Fix For: 1.28.0
The following valid SQL query:
{code:sql}
select "customer_id" as "*" from "customer";
{code}
changes slightly semantics when it is parsed, optimized, and transformed back
to SQL via {{RelToSqlConverter}}.
+Actual+
{code:sql}
SELECT "customer_id" AS """*""" FROM "foodmart"."customer";
{code}
Note the extra quotes appearing in the {{AS}} clause. Aliasing columns
differently is undesirable and can have an impact when this query pattern
appears as part of a larger query and there are usages of the alias.
+Expected+
{code:sql}
SELECT "customer_id" AS "*" FROM "foodmart"."customer";
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Deduplicate correlate variables question.
Thanks, yes, that’s better. > On Jun 30, 2021, at 10:34 PM, stanilovsky evgeny > wrote: > > thanks! done, i hope ) > >> Thanks! Can you improve the JIRA subject, so that everyone reading the >> release notes will understand it? (I had to look up COLLECTION_TABLE.) >> >> >>> On Jun 30, 2021, at 5:12 AM, stanilovsky evgeny >>> wrote: >>> >>> Guys, i fill the ticket [1] and PR is ready for check, can anyone take a >>> look on it ? >>> Additional test appended, all tests locally passed. >>> >>> [1] https://issues.apache.org/jira/browse/CALCITE-4673 >>> >>> thanks ! >>> > The PR needs to fix a bug or implement a feature. So, first you should > log a JIRA case describing what doesn’t work. Write tests for what > doesn’t work that you want to make work. (Or maybe you can > refactor/generalize existing tests.) Then submit a PR, and we will review > that PR on the basis of whether it fixes the bug. > > That may sound like a lot of process. But the alternative is people just > randomly changing code. No, this sounds like a proper way ) Ok, i understand you ! > >> On Jun 3, 2021, at 7:37 AM, stanilovsky evgeny >> wrote: >> >> Julian, thanks for reply and comments. >> Can you explain, is it would possible to commit PR containing >> deduplication code moved upper this flag ? >> If so - i will create PR and rerun all existing tests, of course. >> Thanks ! >> >>> Master is a moving target. Apparently you are using a version of the >>> code from sometime in March. These links work better: >>> >>> [1] >>> https://github.com/apache/calcite/blob/796675c9b33e0461bc45a72780162d474a4b098b/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 >>> >>> [2] >>> https://github.com/apache/calcite/blob/796675c9b33e0461bc45a72780162d474a4b098b/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 >>> >>> I don't know the exact cause of what you are seeing, but when you set >>> "expand=false" you are choosing a newer (and better) code path. We do >>> not have feature parity between the code paths. Nor do we write tests >>> for both code paths. >>> >>> Some day I'd like to officially deprecate, then obsolete, >>> "expand=true". It is de facto deprecated because we put more effort >>> into the "expand=false" path. >>> >>> Julian >>> >>> >>> On Fri, May 28, 2021 at 1:19 AM stanilovsky evgeny >>> wrote: Hi, calciters ) I am trying to figure out why DeduplicateCorrelateVariables [1] is called only if withExpand [2] flag is true ? Why we don`t need to deduplicate in appropriate case ? [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 [2] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 Thanks !
