"FunctionJoin" advice

2024-01-30 Thread Thomas Rynne
I have created a RelNode "FunctionJoin" that takes in input relation and the name of a java method to call to create new rows from an existing row (Array[Any] => Array[Array[Any]]) It's like a join but the rows of right relation are created on demand using the rows of the left relation. I

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Mihai Budiu
I think we should start a new thread. Mihai From: Stephen Carlin Sent: Tuesday, January 30, 2024 5:19 PM To: dev@calcite.apache.org Subject: Re: Storing RexLiteral as BigDecimal values I don't know if this is hijacking or adding to the discussion, but figured

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Stephen Carlin
I don't know if this is hijacking or adding to the discussion, but figured I'd write here. I'm currently implementing Calcite on my database, but I'm actually running into a problem where my db would rather have exact types on some of the lower integers. LIke, if we have the value 38, I need

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Mihai Budiu
Highly relevant discussion: https://issues.apache.org/jira/browse/CALCITE-2067 This change would indeed enable representing other DOUBLE values which cannot be represented by BigDecimal, such as NaN, Infinities, and -0. It looks like the evaluator does not reduce such expressions. That issue

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Mihai Budiu
* Regarding [1], it looks like the power function implementations with DECIMAL arguments return DOUBLE. Where the BigDecimal Java type has custom functions that have potentially higher accuracy than FP versions, using them could be beneficial. Related to this, Calcite uses POWER to represent

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Julian Hyde
In the case of POWER, Postgres has both a decimal and double version[1], and Calcite should do the same. However, I take your point that if the function returns a double, then constant-reduction should create a double literal. I am concerned that developers will accidentally create double

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Mihai Budiu
Well, irrespective of the type of the arguments, the POWER function returns a double value. These are the 4 implementations in SqlFunctions: public static double power(double b0, double b1) { return Math.pow(b0, b1); } public static double power(double b0, BigDecimal b1) { return

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Julian Hyde
It’s surprising to me that 1004.3e0 should have type DOUBLE or REAL; I would expect it to have type DECIMAL(5, 1), where it can be represented exactly. > On Jan 30, 2024, at 2:53 PM, Mihai Budiu wrote: > > e0 shows that these are DOUBLE values. > Moreover, power returns a DOUBLE value. > In FP

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Mihai Budiu
e0 shows that these are DOUBLE values. Moreover, power returns a DOUBLE value. In FP the result is the wrong one, but that's the semantics of the power function in FP. Mihai From: Julian Hyde Sent: Tuesday, January 30, 2024 2:50 PM To: dev@calcite.apache.org

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Julian Hyde
The inputs are decimals, and the correct answer is 1008618.49, also a decimal, and cannot be exactly represented as a binary floating point. I’m not sure why in this case you want a binary floating point. > On Jan 30, 2024, at 2:46 PM, Mihai Budiu wrote: > > I am evaluating this expression:

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Mihai Budiu
I am evaluating this expression: SELECT power(1004.3e0, 2e0) The result in Java, or Postgres, when formatted as a string, is 1008618.48 The result produced by the Calcite simplification code is 1008618.49 The simplification code can produce RexLiterals - that's where this would be

Re: Storing RexLiteral as BigDecimal values

2024-01-30 Thread Julian Hyde
Can you give a scenario where a RexLiteral should have a double value? > On Jan 30, 2024, at 2:36 PM, Mihai Budiu wrote: > > Hello, > > I have a question about the representation of RexLiteral values. > Currently DOUBLE-valued literals are represented using a BigDecimal. > This causes small

Storing RexLiteral as BigDecimal values

2024-01-30 Thread Mihai Budiu
Hello, I have a question about the representation of RexLiteral values. Currently DOUBLE-valued literals are represented using a BigDecimal. This causes small rounding errors, introduced in the RexBuilder.clean() function. This causes FP expressions that are evaluated at compilation-time to

Re: Hosting Tableau Connector in Calcite

2024-01-30 Thread Jerin John
Hi Julian, Thank you for the response and sharing this information! That makes sense, I will draft a PR linked to the JIRA that upstreams the connector files hosted in our looker-open-source Calcite fork [1]. With respect to testing, it currently requires us to set up a Tableau instance with the

Re: About SqlFunctions.java slimming in calcite

2024-01-30 Thread Julian Hyde
I don’t think that SqlFunctions has a problem. Using my IDE I can navigate to any of the functions using a few keystrokes. Because I never needed to scroll through the file, I never noticed how many lines it had. Occasionally people add methods because they think that every SQL function needs

[jira] [Created] (CALCITE-6235) MAP_CONTAINS_KEY function results correspond to Spark results

2024-01-30 Thread Caican Cai (Jira)
Caican Cai created CALCITE-6235: --- Summary: MAP_CONTAINS_KEY function results correspond to Spark results Key: CALCITE-6235 URL: https://issues.apache.org/jira/browse/CALCITE-6235 Project: Calcite

[jira] [Created] (CALCITE-6234) Add Test on SqlOperatorTest for format_date function(enable Bigquery and pg)

2024-01-30 Thread Caican Cai (Jira)
Caican Cai created CALCITE-6234: --- Summary: Add Test on SqlOperatorTest for format_date function(enable Bigquery and pg) Key: CALCITE-6234 URL: https://issues.apache.org/jira/browse/CALCITE-6234

Re: About SqlFunctions.java slimming in calcite

2024-01-30 Thread Cancai Cai
Hi Forward Xu, I think this is a good idea. This can not only improve the readability and maintainability of the code, but it may also be helpful for new developers to learn. However, how to migrate safely is a big problem, because we cannot influence User upgrade Maybe we should start with a

[jira] [Created] (CALCITE-6233) Support parse of ISOWEEK on BigQuery library

2024-01-30 Thread Rodrigo Rueda (Jira)
Rodrigo Rueda created CALCITE-6233: -- Summary: Support parse of ISOWEEK on BigQuery library Key: CALCITE-6233 URL: https://issues.apache.org/jira/browse/CALCITE-6233 Project: Calcite Issue

Re: EnumerableProject question

2024-01-30 Thread Stamatis Zampetakis
Hey Eric, EnumerabeProject and EnumerableFilter are not meant to appear in the plan because they cannot be implemented; as you observed the implement method throws an exception. For this reason it does not make sense to use ENUMERABLE_PROJECT_RULE and ENUMERABLE_FILTER_RULE and this is why the