Re: CoreFilters.FILTER_REDUCE_EXPRESSIONS issue

2024-05-21 Thread Julian Hyde
You say that SELECT code, CASE WHEN code = 'test_val' THEN ABS(val) ELSE NULL END AS absval FROM TEST WHERE CASE WHEN code = 'test_val' THEN ABS(val) > 0 ELSE FALSE END isn't valid SQL. What's wrong with it? It looks valid and correct to me. (I acknowledge that it's neither pretty nor

Re: Apache at Visa Summit

2024-05-14 Thread Julian Hyde
Julian Hyde wrote: > > Visa is a major sponsor of the ASF and is hosting a one-day conference > in Foster City, CA, on Thursday (May 16th) to showcase Apache projects > [1]. I am giving a talk "Open Source Software Innovation in Advanced > Databases and Analytics" [2]. &

Apache at Visa Summit

2024-05-14 Thread Julian Hyde
Visa is a major sponsor of the ASF and is hosting a one-day conference in Foster City, CA, on Thursday (May 16th) to showcase Apache projects [1]. I am giving a talk "Open Source Software Innovation in Advanced Databases and Analytics" [2]. I believe registration is free for ASF committers, and

[jira] [Created] (CALCITE-6408) Not-null ThreadLocal

2024-05-10 Thread Julian Hyde (Jira)
Julian Hyde created CALCITE-6408: Summary: Not-null ThreadLocal Key: CALCITE-6408 URL: https://issues.apache.org/jira/browse/CALCITE-6408 Project: Calcite Issue Type: Improvement

Re: Custom non-sql data types?

2024-05-09 Thread Julian Hyde
Are you sure that you need your constant values to be instances of RexLiteral? A call to a 'constructor function' might do just as well. In Calcite's geospatial support, there is no literal for the GEOMETRY type. Expressions such as ST_PointFromText('POINT(-71.064544 42.28787)') are commonplace

Re: About calcite’s processing of metadata information

2024-05-08 Thread Julian Hyde
Do you know which metadata? For validation it is mostly using interface Schema (by which it accesses table definitions and their columns). For optimization it might use other metadata such as existence of keys, row counts, etc. There are various implementations of Schema - including with

Re: [VOTE] Release Apache Calcite 1.37.0 (release candidate 4)

2024-04-30 Thread Julian Hyde
It seems that Sergey has logged https://issues.apache.org/jira/browse/CALCITE-6393 for this. Please add further discussion to that case rather than to this vote thread. > On Apr 30, 2024, at 12:44 AM, Ruben Q L wrote: > > Thanks Guillaume for checking this! > When I looked at this issue on

Re: Is it posible to change synthetic input names ($0, $1, etc) in explain plans?

2024-04-30 Thread Julian Hyde
Gonzalo, I think the implementation is fairly straightforward. You’d write a new implementation of RelWriter. But, as Alessandro says, the specification is the hard part. What *exactly* would you want instead of $0? (Those fields have names, and most of the time they are the same as the user

Re: Some tests under SqlValidatorTest are disabled with strange annotation

2024-04-26 Thread Julian Hyde
Dtbug 280 is in the long-defunct bug database of a long-defunct company. The comments in calcite code are all that’s left of it. If you can reenable those tests you can safely declare the bug fixed. Julian PS Ironically, “dt” stands for “disruptive tech”. (“My name is Ozymandias, King of

Re: [jira] [Created] (CALCITE-6363) Introduce a rule to derive more filters from inner join condition

2024-04-15 Thread Julian Hyde
James, Thanks for chiming in. I added your comments to the jira case. Julian > On Apr 15, 2024, at 12:58 PM, James Starr wrote: > > The keyword I think you want is transitive filter pushdown. The reduce > expression rule handles some of the trivial cases outlined as examples. > Also, you

Re: Supporting ASOF JOIN

2024-04-15 Thread Julian Hyde
I would regard this as two separate but related things: a new SQL syntax for joins, and a new relational operator. It is definitely worth keeping them separate; the operator will not map 1-1 to the syntax, may require its input to input to be sorted, and of course we would want queries to be

Re: Draft: board report for 2024 Q1

2024-04-03 Thread Julian Hyde
+1 Thank you. Great idea to mention the new ASF members. Julian > On Apr 3, 2024, at 3:11 AM, Benchao Li wrote: > > Hello, > > Below you can find a draft of this quarter's board report. I plan to > submit the final version next Tuesday (Apr 9, 2024). > > Please let me know if you have any

Re: Using quidem tests to validate optimization rules

2024-03-18 Thread Julian Hyde
You are correct. There is no mechanism, and one would be really useful. > On Mar 18, 2024, at 1:19 PM, Mihai Budiu wrote: > > Hello, > > I was looking around to see whether it's possible to use a quidem test to > exercise specific optimization rules, but I couldn't find anything. Is there >

Re: Supporting a function with different capabilities depending on database

2024-03-16 Thread Julian Hyde
is available, then tries to resolve arguments and > return values. > > Thanks for the help. > > > From: Julian Hyde > Date: Monday, March 11, 2024 at 5:08 PM > To: dev@calcite.apache.org > Subject: Re: Supporting a function with different capabilities depending on > datab

Re: Incorrect Quidem test in agg.iq

2024-03-16 Thread Julian Hyde
I started investigating but didn't get a chance to follow up. I don't know whether Calcite or Postgres is correct. I suspect the difference is due to the expression 'comm is null'. Because I don't know whether GROUPING_ID's behavior is defined if its arguments are not all columns. But I think the

Re: Supporting a function with different capabilities depending on database

2024-03-11 Thread Julian Hyde
"...different capabilities depending on database” What exactly do you mean by “database”? If I create a Calcite model that has one schema backed by a Postgres JDBC catalog and another schema backed by a MySQL JDBC catalog, what “database” would you expect a connection to this model to have?

Re: Pull Request Reviews for Avatica

2024-03-11 Thread Julian Hyde
Mihai, Can you list the PRs (jira cases) in this category? I thought I suggested a solution. disable tests in Calcite based on the Avatica version number. And make the changes just before an Avatica release, and quickly upgrade Calcite, so that any tests you need to disable aren’t disabled

Re: Override built-in function signature for query parsing

2024-03-05 Thread Julian Hyde
The parser does its work without looking at the operator table. (Like any sane parser, it deals only with syntax, not semantics.) So, it’s impossible for your TRIM function to affect what the parser considers to be valid. One option is for you to modify the Babel parser, which is a more lenient

Re: Reviewing blog repository

2024-03-05 Thread Julian Hyde
The web site has a ’news’ section which could be used for blog posts. I think it would be great to add this kind of content to the site (copy-pasted from your blog, and with a link to the original blog). People would find it more easily, and it would trust it because it is linked from the

Re: [ANNOUNCE] Sergey Nuyanzin joins Calcite PMC

2024-03-05 Thread Julian Hyde
Welcome, Sergey! Thank you for your efforts over many years in and around the Calcite community (including in SQLLine and Flink) making Calcite better and the community more friendly and efficient. Julian > On Mar 5, 2024, at 5:59 AM, Michael Mior wrote: > > Congratulations and welcome

Re: Simulataneous changes to Avatica and Calcite

2024-02-26 Thread Julian Hyde
In https://github.com/apache/calcite/commit/614b4350f558a4fabaafc06fef9689ec4c267994 you'll see the Stamatis upgraded Avatica and removed the "!if (fixed.calcite2776) {" wrapper in a Quidem test. That is a reasonably clean way to comment out tests pending a bug fix or an upgrade. Calcite release

Re: [DISCUSS] Ensuring that Calcite is consistent with other SQL systems

2024-02-26 Thread Julian Hyde
rc/main/java/org/testcontainers/containers/PostgisContainerProvider.java > https://testcontainers.com/modules/postgis/ > > Thanks > Hanu > > On Mon, Feb 26, 2024 at 11:20 AM Julian Hyde wrote: > > > One more thing. The geospatial library, including Bertil's proposed >

[jira] [Created] (CALCITE-6281) Add test infrastructure to ensure that Calcite is consistent with the SQL dialects it is trying to emulate

2024-02-26 Thread Julian Hyde (Jira)
Julian Hyde created CALCITE-6281: Summary: Add test infrastructure to ensure that Calcite is consistent with the SQL dialects it is trying to emulate Key: CALCITE-6281 URL: https://issues.apache.org/jira/browse

Re: [DISCUSS] Ensuring that Calcite is consistent with other SQL systems

2024-02-26 Thread Julian Hyde
rt PostGIS? Julian [5] https://issues.apache.org/jira/browse/CALCITE-6239 On Mon, Feb 26, 2024 at 11:07 AM Julian Hyde wrote: > > Thank you, everyone for your thoughts so far. (And let's keep talking!) > > I don't yet see a full, perfect solution. But I see a number of > techniques/tool

Re: [DISCUSS] Ensuring that Calcite is consistent with other SQL systems

2024-02-26 Thread Julian Hyde
Thank you, everyone for your thoughts so far. (And let's keep talking!) I don't yet see a full, perfect solution. But I see a number of techniques/tools we can use. For example, I was not aware of testContainers but it seems to very quickly address our most common use cases (MySQL + Postgres).

Re: [DISCUSS] Ensuring that Calcite is consistent with other SQL systems

2024-02-25 Thread Julian Hyde
. On Sun, Feb 25, 2024 at 12:25 PM Guillaume Masse wrote: > > Write test with SQL logic test > > https://www.sqlite.org/sqllogictest/doc/trunk/about.wiki > > Then run those tests against each database. (This technique is called > oracle testing) > > > On Sun, Feb 25, 20

[DISCUSS] Ensuring that Calcite is consistent with other SQL systems

2024-02-25 Thread Julian Hyde
TL;DR: We need to add software engineering processes to ensure that Calcite is consistent with other SQL systems. What should those be? There has been a lot of activity recently adding functions such as LOG2 [1] and ensuring that existing functions such as LOG and SQRT [2] are consistent with

Re: Remove a specific type coersion rule

2024-02-23 Thread Julian Hyde
Mihai Budiu () escribió: > >> When I filed the Jira issue I was planning to give an error, but I >> discovered that several dialects implement this cast, so my PR actually >> implements the behavior from Postgres. >> >> I will amend the Jira case when we decide what t

Re: [DISCUSS] Towards Calcite 1.37.0

2024-02-23 Thread Julian Hyde
rrently 100+ new commits in main branch. Per our >>> tradition of producing one release every 2-3 months, I think it's time >>> to consider for next release now. >>> >>> According to [2], the following release managers would be: >>> - 1.37.0 Sergey Nuya

Re: about optimization rules for over(partition by id, age order by id, age)

2024-02-22 Thread Julian Hyde
I noticed that DuckDB implemented such an optimization recently. Can you please log a jira case to remove redundant ORDER BY keys from OVER? Yes, it would be either a planner rule or a rewrite by RelBuilder. Julian > On Feb 21, 2024, at 11:43 PM, key lou wrote: > > HI ALL : > I have the

Re: Some questions about calcite

2024-02-22 Thread Julian Hyde
eparate log2 function that is > undefined for 0 and negative values (i.e., can return any value for such > arguments), let's just not pretend it's the MySQL function. > > Mihai > ____ > From: Julian Hyde > Sent: Thursday, February 22, 2024 4:05 PM > To

Re: Remove a specific type coersion rule

2024-02-22 Thread Julian Hyde
When I read CALCITE-6210 it wasn’t clear that you wanted Calcite to give a validation error when someone tries to cast a VARCHAR to a VARBINARY. I agree that that is the desirable behavior. (And I believe it is consistent with the SQL standard.) Can you amend the Jira case so that it clearly

Re: Some questions about calcite

2024-02-22 Thread Julian Hyde
> let bugs in deliberately. They may essentially never get fixed. > > I don't think a compiler can cut any corners. The compiler is the foundation > of an entire software ecosystem. If the foundation is broken, everything > crumbles. > > Mihai > > __

Re: Some questions about calcite

2024-02-22 Thread Julian Hyde
Don’t let the perfect be the enemy of the good. Or as they say in open source, “Release early and often”. Just about everyone who wants a LOG2 function is intending to apply it to positive numbers. So they won’t notice, or care, that the function doesn’t do exactly what they expected when you

Re: What's the necessity of the HepRelVertex used in HepPlanner

2024-02-21 Thread Julian Hyde
Let’s suppose that the information in HepRelVertex could somehow be moved into the RelNode. (Since the data structure is a DAG it seems plausible.) Now each RelNode would contain mutable state that relates to the HEP planner’s algorithm. That state needs to be managed (e.g. reinitialized when

Re: Review request.

2024-02-19 Thread Julian Hyde
Please respond to my comments in Jira. It's difficult to review a PR until we know what problem it is fixing. On Mon, Feb 19, 2024 at 5:53 PM Hanumath Maduri wrote: > > Hello Developers, > > I've had a pull request (https://github.com/apache/calcite/pull/3640) open > for review over the past few

Re: ** Can we pass constructor while registering UDF's **

2024-02-19 Thread Julian Hyde
The best way to figure out what is possible/supported is to read the unit tests. E.g. UdfTest.java. On Mon, Feb 19, 2024 at 10:16 AM Akshay wrote: > > Hi > > I was wondering if someone could help me with this. > > I have a custom Avro Adapter that applies SQL on AVRO records. I register > all

Re: Numeric literals

2024-02-14 Thread Julian Hyde
It's not unexpected. :) Would you expect us to require whitespace between one token and the next? If you write '(1+2)' there are five tokens, no whitespace required. By the way, Postgres seems to do the same as Calcite. On Wed, Feb 14, 2024 at 6:13 PM Sean Broeder wrote: > > It looks like

Re: Implementing a SqlDialect for PostGIS

2024-02-12 Thread Julian Hyde
> - The column aliases returned by Calcite are in uppercase, while those > returned by PostGIS through the JDBC adapter are in lowercase. I think this might be due to how the parser handles unquoted identifiers. In Calcite, as in Oracle, if I wrote ’select empno as x from emp’, the unquoted

Re: streaming query window operators

2024-02-09 Thread Julian Hyde
Your proposals will of course work. But they force the user to think in terms of a lower-level language. * TUMBLE is of course simple. Before we had TUMBLE we used "GROUP BY FLOOR(…)”. * HOP is a bit more complicated, because you need to create multiple copies of each row, each going into a

Re: Question about PR approvals

2024-02-08 Thread Julian Hyde
I don’t think of GitHub’s approval check-box as ‘official’. As a committer, you can decide when you have ’sufficient approval’. If it’s a simple change, no approval is needed. Sometimes I say ‘looks good to go, please add javadoc to class Xyz’ and I would not expect (or want) to re-review. >

Re: About Negative testing for adapting to different databases

2024-02-08 Thread Julian Hyde
Functions have slightly different behaviors on different databases. Solving the problem starts with logging a jira case, and having a discussion about the specification. Rather than say (for example) ’SOUNDEX returns the wrong result’, it’s helpful to say ’SOUNDEX result for empty string is

Re: Registering rules from multiple RelNodes of identical type

2024-02-02 Thread Julian Hyde
That method is intended if you have a class MyTableScan and a generic rule that can handle any instance of MyTableScan. If you have (say) five instances of MyTableScan and five rule instances, one for each scan, then you should register each rule instance individually in the planner. There

Re: Storing RexLiteral as BigDecimal values

2024-01-31 Thread Julian Hyde
for REAL) to store values > either as BigDecimal, which is suitable when the literals come from the > parser, or as Double, which is suitable when values come from the evaluator. > I plan to file a JIRA issue to enable this feature. > > Thank you, > Mihai > > ___

Re: "FunctionJoin" advice

2024-01-31 Thread Julian Hyde
‘Function join’ sounds similar to ‘CROSS APPLY’ - for each row on the left, call a function to generate a list of rows on the right. Is that how you see it? If so, there are some beautiful parallels between CROSS APPLY and lateral join; for example, they can both be de-correlated. See [1] and

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Julian Hyde
> public static double power(BigDecimal b0, BigDecimal b1) { >return Math.pow(b0.doubleValue(), b1.doubleValue()); > } > > The problem is that, for the query below, the optimizer will then convert the > double result of power into a BigDecimal, rounding it in the process

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Julian Hyde
lue. > In FP the result is the wrong one, but that's the semantics of the power > function in FP. > > Mihai > ________ > From: Julian Hyde > Sent: Tuesday, January 30, 2024 2:50 PM > To: dev@calcite.apache.org > Subject: Re: Storing RexLiter

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Julian Hyde
l. > This rounding error is not really necessary. > > Mihai > ____ > From: Julian Hyde > Sent: Tuesday, January 30, 2024 2:40 PM > To: dev@calcite.apache.org > Subject: Re: Storing RexLiteral as BigDecimal values > > Can you give a

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Julian Hyde
Can you give a scenario where a RexLiteral should have a double value? > On Jan 30, 2024, at 2:36 PM, Mihai Budiu wrote: > > Hello, > > I have a question about the representation of RexLiteral values. > Currently DOUBLE-valued literals are represented using a BigDecimal. > This causes small

Re: About SqlFunctions.java slimming in calcite

2024-01-30 Thread Julian Hyde
I don’t think that SqlFunctions has a problem. Using my IDE I can navigate to any of the functions using a few keystrokes. Because I never needed to scroll through the file, I never noticed how many lines it had. Occasionally people add methods because they think that every SQL function needs

Re: Hosting Tableau Connector in Calcite

2024-01-29 Thread Julian Hyde
Thanks for reaching out, Jerin! (Jerin and I work together at Google, but I’m wearing my Calcite hat here.) A Tableau connector is something Calcite has needed for some time, and in fact there is a Jira case for it [1]. The goals were (1) create a 'vanilla' connector to that Tableau can connect

Re: Thesis question regarding Calcite placement in taxonomy

2024-01-29 Thread Julian Hyde
The taxonomy seems about data stores (one or many) and supported query languages (one or many). Calcite is ‘many’ in each category, even though it appears to be one query language (SQL) and no data stores. TL;DR: I would put it in the ‘polystore’ category. “Query answering” is one of its

Re: Index Based QueryableTable Implementation

2024-01-26 Thread Julian Hyde
There is class AbstractQueryable so you shouldn’t need to implement the whole interface by hand. However, a better next step after FilterableTable is not QueryableTable, but Table with a few planner rules to push down filters, projects, etc. Calcite gets its intelligence from how rules compose

Re: Question on `Expressions#constant` accepting only public field models

2024-01-26 Thread Julian Hyde
It’s reasonable to extend the kind of fields that we allow. You should log a jira case and submit a PR with appropriate tests. It might be tricky to test Java records given that the test suite needs to compile and pass on JDK 8, but maybe you can do something using reflection. Julian > On

Re: calcite 1.36.0 release procedures

2024-01-26 Thread Julian Hyde
JDK, see > similar issues: > https://gitlab.ow2.org/asm/asm/-/issues/317789 > https://bugs.openjdk.org/browse/JDK-8144185 > https://bugs.openjdk.org/browse/JDK-8187805 > > > > On Thu, Dec 21, 2023 at 8:58 PM Julian Hyde wrote: > >> I think I was mistaken ab

Re: [Question] Query Introspection on SqlNode, RexNode, and RelNode

2024-01-26 Thread Julian Hyde
Matthew, For the kinds of tasks that you mention, I tend to use the visitors. For example, to find whether can expression contains an IS NULL operator: public static boolean containsIsNull(RexNode e) { try { e.accept(new RexVisitorImpl(true) { @Override public Void

Re: [Question] Unknown cost calculation/propagation in RelSubsets

2024-01-24 Thread Julian Hyde
; > [1]: https://ieeexplore.ieee.org/abstract/document/344061 > [2]: > https://liuyehcf.github.io/resources/paper/The-Cascades-Framework-For-Query-Optimization.pdf > > Best regards, > Tony > > > Am 16.01.24 um 23:17 schrieb Julian Hyde: >> Tony, >> You’re

Re: Refactor reference.md

2024-01-24 Thread Julian Hyde
> I am not proposing a new process, the existing review process would continue > to apply. The page would still be part of the repository. Just a separate web > page on the calcite site, unbundled from the SQL language page. > > Mihai > > ________ > From:

Re: Refactor reference.md

2024-01-24 Thread Julian Hyde
; attempts to clarify something, but has not been approved since early December. > > Mihai > > From: Julian Hyde > Sent: Wednesday, January 24, 2024 12:54 PM > To: dev@calcite.apache.org > Subject: Re: Refactor reference.md > > Extr

Re: Refactor reference.md

2024-01-24 Thread Julian Hyde
Extra documentation would be nice. But who is going to write (and maintain) this extra documentation? Even the current documentation takes a lot of work. When reviewing a PR to add a function, I have to tell people to remove a ‘.’ at the end of the line to be consistent with the existing doc.

Re: Unparsing based on operand type

2024-01-23 Thread Julian Hyde
It’s not possible to do during front-end phases (validation or sql-to-rel) because you don’t yet know what the target dialect is. Not in general anyway; I know that Calcite is often run as a “translation layer”, where there is a single target dialect, but it’s not a clean architecture to make

Re: [Question] Derived type from string expression with NULL parameter

2024-01-22 Thread Julian Hyde
ug is in the fact that UPPER uses the constructor for >>> SqlFunction (well, almost a constructor) which doesn't supply an operand >>> type inference argument. >>> >>> Still, if the operand type checker requires a CHARACTER family, why is a >>> NULL literal always acc

[jira] [Created] (CALCITE-6217) PER clause for aggregate functions

2024-01-22 Thread Julian Hyde (Jira)
Julian Hyde created CALCITE-6217: Summary: PER clause for aggregate functions Key: CALCITE-6217 URL: https://issues.apache.org/jira/browse/CALCITE-6217 Project: Calcite Issue Type

Re: javadoc in chinese

2024-01-18 Thread Julian Hyde
Do you it’s worth logging a jira case? It’s a bug with symptoms that are visible beyond the Calcite site and current release, so I think a Jira would help document it. Is it possible to fix the 1.36 doc? I don’t believe so. And actually I think it’s rather a neat curiosity, so we should let

[DISCUSS] Does web site use Google analytics?

2024-01-17 Thread Julian Hyde
Does Calcite’s web site use Google analytics? If so, we should fix it. Google analytics collects information that is not allowed by GDPR. Infra is considering blocking Google analytics for all *.apache.org sites, and if they do that it may make our site unusable. Whimsy

Re: [Question] Unknown cost calculation/propagation in RelSubsets

2024-01-16 Thread Julian Hyde
Tony, You’re asking about how the Volcano algorithm computes metadata for equivalence classes (what Calcite calls subsets) and to my knowledge it’s not been spelled out explicitly (either in the Volcano/Cascades papers or in Calcite discussions). Calcite needs various kinds of metadata, such

Re: PR 3495 (Make single-row rels unique) Review Request

2024-01-16 Thread Julian Hyde
> Thanks, > -Paul > > > On Monday, January 8, 2024 at 05:41:30 PM EST, Julian Hyde > wrote: > > > > I see two failures due to checkerframework. Those are real. Let us > know when you've fixed them. > > The failure in Druid is probably due to a Druid bug that

Re: Why are window functions considered agg function?

2024-01-12 Thread Julian Hyde
Simple answer is that an aggregate function is one that takes a set of values. For example, if you apply the SUM function to values [1, 2, 3] it returns 6. When I execute a GROUP BY statement, the values of a particular column within a particular group (e.g. the group of rows for which deptno =

Re: [Question] Derived type from string expression with NULL parameter

2024-01-12 Thread Julian Hyde
OperandTypeChecker just makes sure that arguments of known type are compatible; another interface, SqlOperandTypeInference, infers the types of arguments of unknown type. Hopefully it’s now a little less mysterious. > On Jan 12, 2024, at 11:12 AM, Mihai Budiu wrote: > > inference in Calcite

Re: PR 3495 (Make single-row rels unique) Review Request

2024-01-08 Thread Julian Hyde
I see two failures due to checkerframework. Those are real. Let us know when you've fixed them. The failure in Druid is probably due to a Druid bug that existed in November. You can ignore it. Should go away after rebase. Please keep nagging on this thread... this is a good change, and we should

Re: Draft: board report for 2023 Q4

2024-01-08 Thread Julian Hyde
t; The number of non-committer (contributor) commits per month: >>> +--+---+-+ >>> | year | month | contributor_commits | >>> +--+---+-----+ >>> | 2023 |10 | 35

Re: Alias on union

2024-01-05 Thread Julian Hyde
Maybe the problem is the ‘build()’ after ‘as(“core”)’? The alias is associated with entry on the RelBuilder stack, not the RelNode. Once you have popped it, the alias is forgotten. Julian > On Jan 5, 2024, at 10:47, Eric Berryman wrote: > > I’m trying to use a union in a join, but I don’t

Re: Multi-query optimization

2024-01-04 Thread Julian Hyde
of those optimizations could be implemented in > Calcite, if there are enough extension points to plug in custom logic. > > There's probably nothing new here, but I hope this confirms such > functionality's usefulness. > > Would be happy to help with the design/impl

[jira] [Created] (CALCITE-6188) Multi-query optimization

2024-01-04 Thread Julian Hyde (Jira)
Julian Hyde created CALCITE-6188: Summary: Multi-query optimization Key: CALCITE-6188 URL: https://issues.apache.org/jira/browse/CALCITE-6188 Project: Calcite Issue Type: Improvement

Re: [MINOR]

2024-01-04 Thread Julian Hyde
I have merged https://github.com/apache/calcite/commit/3c19347cc45349a21a8c97d6f6e8d3e9f596070f with a new lint rule. I had to force-push to amended the previous two commits because the linter checks the N most recent commits. Julian > On Jan 4, 2024, at 11:28 AM, Julian Hyde wr

Re: Multi-query optimization

2024-01-04 Thread Julian Hyde
directions require further research and exploration. >> >> Finally here is a related paper MULTI-QUERY OPTIMIZATION AND APPLICATIONS] >> <https://www.cse.iitb.ac.in/infolab/Data/Courses/CS632/2015/2014/Papers/prasan-thesis.pdf> >> . >> >> Best, &

[jira] [Created] (CALCITE-6187) Linter should disallow tags such as '[MINOR]' in commit messages

2024-01-04 Thread Julian Hyde (Jira)
Julian Hyde created CALCITE-6187: Summary: Linter should disallow tags such as '[MINOR]' in commit messages Key: CALCITE-6187 URL: https://issues.apache.org/jira/browse/CALCITE-6187 Project: Calcite

Re: [MINOR]

2024-01-04 Thread Julian Hyde
ery small and there is no related work order or issue, we > can briefly describe the problem and explain the reason for the fix when > submitting the PR. If we need to standardize this type of PR submission, we > can form some PR templates or documentation. > > Best, > ForwardXu >

Re: Feature: Support gremlin adapter

2024-01-03 Thread Julian Hyde
I had a similar question to Mihai. After your change, would Calcite be able to translate a user's SQL query and execute it against an existing implementation of the Gremlin API (say neo4j), or would users be able to talk to Calcite in the Gremlin API and have Calcite convert that, via relational

Re: [MINOR]

2024-01-03 Thread Julian Hyde
gt; > > > > Also check for Gerrit Change IDs which are often added automatically, > > > > and a > > > > paint to remove. > > > > > > > > Istvan > > > > > > > > On Tue, Jan 2, 2024 at 10:50 PM Tanner Clary > >

Multi-query optimization

2024-01-03 Thread Julian Hyde
Multi-query optimization is one of the big challenges of our field. Examples of multi-queries: * an INSERT statement that writes into a table but also updates an index, * a DAG that represents an ETL/ELT job; * a query that produces several data sets (say a list of invoices for orders and a

Re: Nested join support

2024-01-02 Thread Julian Hyde
JOIN ( > VALUES (100, > 'Random item')) AS "lineitem"(l_orderkey, > l_itemname) ON o_orderkey = > l_orderkey ON c_custkey = o_custkey > > > > On Jan 2, 2024, at 1:07 PM, Julian Hyde wrote

Re: Nested join support

2024-01-02 Thread Julian Hyde
What is 'this syntax' you think we should support? (Your query is poorly formatted, so I can't see what pattern in it is confusing the parser.) On Tue, Jan 2, 2024 at 11:46 AM Sean Broeder wrote: > > It looks like Calcite doesn't support the query > > select * from (values (1, 'John')) as > >

[MINOR]

2024-01-02 Thread Julian Hyde
Ralph Waldo Emerson once wrote: “A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." That said, people tend to bring conventions from other projects to Calcite, and we end up with chaos. By which I mean, lots of self-expression, but no

Re: [ANNOUNCE] New Calcite PMC chair: Benchao Li

2023-12-21 Thread Julian Hyde
Congratulations, Benchao. And thank you - for your work over many years making our community kinder and healthier. I know you will be an excellent PMC chair. Thank you, Stamatis, for serving as chair not once but twice. Julian On Thu, Dec 21, 2023 at 6:40 AM Enrico Olivelli wrote: > >

Re: calcite 1.36.0 release procedures

2023-12-21 Thread Julian Hyde
web.asm.ClassReader.accept(ClassReader.java:745) > > at org.objectweb.asm.ClassReader.accept(ClassReader.java:425) > > at remapper.bug.RemapperTest.runTest(RemapperTest.java:53) > > at > > remapper.bug.RemapperTest.calcite35WithCheck(RemapperTest.java:3

Re: calcite 1.36.0 release procedures

2023-12-20 Thread Julian Hyde
ov 24, 2023 at 10:09 PM Guillaume Masse >>> wrote: >>> >>>> We run spark in AWS EMR and it overrides the classpath, so we don't have >>>> control over it: >>>> >>>> Sparks will pull guava 14.0.1: >>>> >>>>

Re: Case-sensitive parser configuration for quoted identifiers

2023-12-19 Thread Julian Hyde
Here’s how I would find out. Find the git commit where those properties were added. Look for tests added in the same commit. Do those tests cover the case you are interested in? Do those tests still work? How do they work? Step through them in the debugger if it’s not clear. > On Dec 18, 2023,

Re: warnings in Calcite

2023-12-19 Thread Julian Hyde
ses where at compile time an exception is thrown and caught could > be warnings - e.g. Casts that will fail at runtime, division by 0, etc. > > Mihai > ____ > From: Julian Hyde > Sent: Monday, December 18, 2023 10:41:11 PM > To: dev@calcite.apach

Re: warnings in Calcite

2023-12-18 Thread Julian Hyde
I don’t think so. One place to surface warnings would be via java.sql.Statement.getWarnings() but Avatica’s default implementation of that method returns null. What are some scenarios where you think Calcite should give a warning? Julian > On Dec 18, 2023, at 7:13 PM, Mihai Budiu wrote: >

Re: Case-sensitive parser configuration for quoted identifiers

2023-12-18 Thread Julian Hyde
Yes, Calcite can do this. In CalciteConnectionProperty [1] there are separate properties QUOTED_CASING and UNQUOTED_CASING. Hopefully you can figure out how to set them. Julian [1] https://calcite.apache.org/javadocAggregate/org/apache/calcite/config/CalciteConnectionProperty.html > On Dec

Re: Support for time zones

2023-12-15 Thread Julian Hyde
Yesterday's XKCD is a propos: https://xkcd.com/2867/ (Depending on your frame of reference and general gravitational situation it may not be YOUR yesterday's XKCD.) On Fri, Dec 15, 2023 at 12:33 PM Julian Hyde wrote: > > I like your approach to divide the work into moderate-sized steps.

Re: Support for time zones

2023-12-15 Thread Julian Hyde
I like your approach to divide the work into moderate-sized steps. Clear descriptions of those steps help those of us who are overwhelmed with email (all of us!) track the situation. The first step seems to be ’Support TIMESTAMP and TIME WITH TIME ZONE in the parser’. (I’ve reversed your

Re: In memory schema and table access to objects

2023-12-13 Thread Julian Hyde
There are limits to what you can do via JDBC. But if you make your implementation of java.sql.ResultSet override the isWrapperFor(Class) and unwrap(Class) methods, you can give access to the internals. Julian [1]

Re: Calcite tables which support multiple data retrieval patterns

2023-12-11 Thread Julian Hyde
PS Austin, You and the people on the cc: list should subscribe to dev@calcite. It will allow you to post without moderation (which often adds a delay of several hours) and will allow you to receive replies. > On Dec 11, 2023, at 12:05 PM, Julian Hyde wrote: > > Thanks for

Re: Calcite tables which support multiple data retrieval patterns

2023-12-11 Thread Julian Hyde
Thanks for starting this conversation. I believe we should be looking for common patterns and devising ways to solve them using relational algebra. (Because that’s the hammer that we have in Calcite… and it’s a very powerful hammer. For example, we can represent all kinds of materialized

Re: Failing CI

2023-12-06 Thread Julian Hyde
wrote: > >> An update on this, it has been solved in CALCITE-6123 by Stamatis, the >> Druid test is supposed to be stable now, thanks Stamatis for fixing >> this and also thanks everyone who helped on this. >> >> Julian Hyde 于2023年12月2日周六 05:11写道: >>

Re: [DISCUSS] BEARER authentication support in Avatica - 2nd try

2023-12-04 Thread Julian Hyde
I saw the discussion; it was not marked as spam. I am not familiar with BEARER authentication so I didn’t reply. If it is indeed a common requirement, and people are prepared to do the work, I would be delighted to have this feature. Julian > On Dec 3, 2023, at 9:45 PM, Istvan Toth wrote: >

Failing CI

2023-12-01 Thread Julian Hyde
CI has been failing intermittently for three weeks. In 9 of the last18 commits to main, the Druid test has failed. See all the red ‘x’ symbols on https://github.com/apache/calcite/commits/main. This has a been logged as https://issues.apache.org/jira/browse/CALCITE-6123, and there has been

Re: avatica-go: resultset offset management

2023-11-27 Thread Julian Hyde
Egor, I don’t know whether your hypothesis is correct, but it sounds plausible and well-researched. Therefore I think you should put it into a jira case, so the issue and discussion is on the record. If there is a flaw in your argument and the jira case is subsequently closed, there’s no shame

  1   2   3   4   5   6   7   8   9   10   >