Some tests under SqlValidatorTest are disabled with strange annotation
Hi !
I see that some tests are muted under strange annotation:
@Disabled("all collation-related tests are disabled due to dtbug 280")
who know what does it mean and probably there is corresponding CALCITE
issue exist ?
thanks !
Re: [Question] Derived type from string expression with NULL parameter
got it, thanks for clarifications.
There is no clear spec of what the type should be, so it's hard to say
whether this is a bug. The NULL type that is inferred can be interpreted
as correct given the context, that expression indeed will evaluate to
NULL. It is more specific than VARCHAR, though. But if this function is
wrong, many other functions are wrong.
Mihai
From: stanilovsky evgeny
Sent: Sunday, January 14, 2024 11:57 PM
To: dev@calcite.apache.org
Subject: Re: [Question] Derived type from string expression with NULL
parameter
Thanks for reply, so can i interpret it as a bug and fill an issue ?
So maybe the bug is in the fact that UPPER uses the constructor for
SqlFunction (well, almost a constructor) which doesn't supply an operand
type inference argument.
Still, if the operand type checker requires a CHARACTER family, why is a
NULL literal always accepted?
Mihai
From: Julian Hyde
Sent: Friday, January 12, 2024 11:52 AM
To: dev@calcite.apache.org
Subject: Re: [Question] Derived type from string expression with NULL
parameter
OperandTypeChecker just makes sure that arguments of known type are
compatible; another interface, SqlOperandTypeInference, infers the types
of arguments of unknown type.
Hopefully it’s now a little less mysterious.
On Jan 12, 2024, at 11:12 AM, Mihai Budiu wrote:
inference in Calcite is still mysterious to me.
Here it infers the type of the argument of UPPER to be NULL instead of
VARCHAR.
That's why the result type of UPPER is also NULL.
The fact that UPPER has an OperandTypeChecker of
"OperandTypes.CHARACTER" is completely ignored:
public static final SqlFunction UPPER =
SqlBasicFunction.create("UPPER",
ReturnTypes.ARG0_NULLABLE,
OperandTypes.CHARACTER,
SqlFunctionCategory.STRING);
Mihai
________
From: stanilovsky evgeny
Sent: Friday, January 12, 2024 4:23 AM
To: dev@calcite.apache.org
Subject: [Question] Derived type from string expression with NULL
parameter
Plz help me to understand is it a bug ?
From standard:
6.18
...cut...
::= { UPPER | LOWER }
6)If is specified, then:
a) The declared type of the result of is the declared type of
the
!!.
thus i expect that return type from: SELECT UPPER(null) will be VARCHAR
but calcite show :
SqlValidatorTest
@Test void testTypeOfUpper() {
sql("SELECT UPPER(NULL)")
.columnType("???");
}
Expected: is "???"
but: was "NULL"
I think it need to be fixed ?
Thanks !
Re: [Question] Derived type from string expression with NULL parameter
Thanks for reply, so can i interpret it as a bug and fill an issue ?
So maybe the bug is in the fact that UPPER uses the constructor for
SqlFunction (well, almost a constructor) which doesn't supply an operand
type inference argument.
Still, if the operand type checker requires a CHARACTER family, why is a
NULL literal always accepted?
Mihai
From: Julian Hyde
Sent: Friday, January 12, 2024 11:52 AM
To: dev@calcite.apache.org
Subject: Re: [Question] Derived type from string expression with NULL
parameter
OperandTypeChecker just makes sure that arguments of known type are
compatible; another interface, SqlOperandTypeInference, infers the types
of arguments of unknown type.
Hopefully it’s now a little less mysterious.
On Jan 12, 2024, at 11:12 AM, Mihai Budiu wrote:
inference in Calcite is still mysterious to me.
Here it infers the type of the argument of UPPER to be NULL instead of
VARCHAR.
That's why the result type of UPPER is also NULL.
The fact that UPPER has an OperandTypeChecker of
"OperandTypes.CHARACTER" is completely ignored:
public static final SqlFunction UPPER =
SqlBasicFunction.create("UPPER",
ReturnTypes.ARG0_NULLABLE,
OperandTypes.CHARACTER,
SqlFunctionCategory.STRING);
Mihai
________
From: stanilovsky evgeny
Sent: Friday, January 12, 2024 4:23 AM
To: dev@calcite.apache.org
Subject: [Question] Derived type from string expression with NULL
parameter
Plz help me to understand is it a bug ?
From standard:
6.18
...cut...
::= { UPPER | LOWER }
6)If is specified, then:
a) The declared type of the result of is the declared type of the
!!.
thus i expect that return type from: SELECT UPPER(null) will be VARCHAR
but calcite show :
SqlValidatorTest
@Test void testTypeOfUpper() {
sql("SELECT UPPER(NULL)")
.columnType("???");
}
Expected: is "???"
but: was "NULL"
I think it need to be fixed ?
Thanks !
[Question] Derived type from string expression with NULL parameter
Plz help me to understand is it a bug ?
From standard:
6.18
...cut...
::= { UPPER | LOWER }
6)If is specified, then:
a) The declared type of the result of is the declared type of the
!!.
thus i expect that return type from: SELECT UPPER(null) will be VARCHAR
but calcite show :
SqlValidatorTest
@Test void testTypeOfUpper() {
sql("SELECT UPPER(NULL)")
.columnType("???");
}
Expected: is "???"
but: was "NULL"
I think it need to be fixed ?
Thanks !
Re: Minimum Guava version
+1, let`s make ! For the upcoming release 1.36, the default Guava version (and the version we compile with) will change from 19.0 to the latest, 32.1.3-jre [1]. (We were on 19.0 because Druid needed to use 19.0, and due to some API changes in Guava, code compiled on 20 or higher cannot be run on 19.0.) As before, we will support (and test) a wide range of Guava versions. But what should be the minimum supported version? I vote for 21.0 [2] because that was the release where Guava introduced support for JDK 8 features - streams, Supplier. We will be able to remove '::get' from a few places, and stop using the Util.toImmutableList() collector. I don't feel strongly about this, but if literally no one needs Guava 19 or 20 anymore let's make the break. Julian [1] https://issues.apache.org/jira/browse/CALCITE-5763 [2] https://github.com/google/guava/wiki/Release21
Re: CALCITE-5678/CALCITE-5957: Datetime literal validation
Thank you, Stamatis. I vote for direct following the SQL standard, i.e.: only strict SQL standard compliant literals date like 10-1-1 is strange and contain huge field for human errors. Hey everyone, CALCITE-5678, which landed recently in Avatica, enforces strict validation of datetime literals and is inline with the SQL standard specification. However, this strict validation also leads to behavior changes (e.g., CALCITE-5957) since some previously "valid" dates are now considered invalid. Roughly four people have expressed an opinion on the aforementioned JIRAs but since this is a change that will likely have wider impact I would prefer to gather some more inputs from the community. I created a very brief anonymous survey [1] with a few sample literals to aid the decision on what Calcite/Avatica should consider valid and invalid from now onwards. I will close the survey in 96 hours from now. Please fill in the survey and/or comment under the respective tickets. Based on the feedback we can opt to do one of the following: * Revert CALCITE-5678 (restore the liberal parsing of datetime literals) * Merge CALCITE-5957 (stricter parsing but leading zeros are optional) * Reject CALCITE-5957 (only strict SQL standard compliant literals) Best, Stamatis [1] https://forms.gle/23f3yVhJCKZJdqQu8
Re: [jira] [Created] (CALCITE-5950) Default column constraint is erroneously processed.
search for reviewer, can anyone plz ? Guys, plz help with review ? Need >=1 additional eyes ) Julian thanks. PR passed all checks and ready for review. I just took a look at https://github.com/apache/calcite/pull/3393/checks. I see an autostyle violation, and it looks valid. That error is there so that you can fix it and not waste reviewers' time. On Wed, Aug 23, 2023 at 2:15 PM Julian Hyde wrote: There aren't many false positives or flaky tests in CI, checker-framework, error-prone. If it says that (say) you are passing a nullable object to a method that requires a not-nullable argument, you probably are. I find it useful to run checker-framework on my own computer, rather than waiting for CI. But you must use JDK 11: ./gradlew --no-parallel --no-daemon -PenableCheckerframework :linq4j:classes :core:classes Julian On Wed, Aug 23, 2023 at 8:42 AM stanilovsky evgeny wrote: > > PR is ready for review, plz explain did i need to fix all of: CI / > CheckerFramework issues ? > Seems some of them are false positive. > thanks ! > > > Evgeny Stanilovsky created CALCITE-5950:
SqlValidatorImpl#deriveAlias no called any more after refactoring
After refactoring [1] merged between 1.32 and 1.34 ver. SqlValidatorImpl#deriveAlias is not called\used any more from calcite code, thus no one can overwrite it for custom purpose. I decide this is erroneous and need to be fixed. WDYT ? Thanks ! [1] git log --grep='Refactor: Deprecate SqlValidatorUtil.getAlias' commit 5d671ec976f748869481d440c168bea45356216e Author: Julian Hyde Date: Sun Oct 31 23:34:15 2021 -0700 Refactor: Deprecate SqlValidatorUtil.getAlias
Re: [jira] [Created] (CALCITE-5950) Default column constraint is erroneously processed.
Guys, plz help with review ? Need >=1 additional eyes ) Julian thanks. PR passed all checks and ready for review. I just took a look at https://github.com/apache/calcite/pull/3393/checks. I see an autostyle violation, and it looks valid. That error is there so that you can fix it and not waste reviewers' time. On Wed, Aug 23, 2023 at 2:15 PM Julian Hyde wrote: There aren't many false positives or flaky tests in CI, checker-framework, error-prone. If it says that (say) you are passing a nullable object to a method that requires a not-nullable argument, you probably are. I find it useful to run checker-framework on my own computer, rather than waiting for CI. But you must use JDK 11: ./gradlew --no-parallel --no-daemon -PenableCheckerframework :linq4j:classes :core:classes Julian On Wed, Aug 23, 2023 at 8:42 AM stanilovsky evgeny wrote: > > PR is ready for review, plz explain did i need to fix all of: CI / > CheckerFramework issues ? > Seems some of them are false positive. > thanks ! > > > Evgeny Stanilovsky created CALCITE-5950:
Re: LITERAL_AGG question
Thanks, i fill the issue.
https://issues.apache.org/jira/browse/CALCITE-5969
First off, can you create a jira case? This is clearly a bug.
I think AggregateCall.rexList is new and will need to be explicitly
handled in serialization and deserialization. Testing shouldn’t be
difficult if you extend or modify existing RelJson tests.
On Aug 28, 2023, at 7:30 AM, stanilovsky evgeny
wrote:
hello, please can someone explain me how LITERAL_AGG need to be
correctly serialized\deserealized and how to write appropriate tests.
Problem :
we have LITERAL_AGG
further:
toJson(org.apache.calcite.rel.core.AggregateCall) {
Map map = map();
map.put("agg", toJson(node.getAggregation()));
map.put("type", toJson(node.getType()));
map.put("distinct", node.isDistinct());
map.put("operands", node.getArgList());
map.put("filter", node.filterArg);
map.put("name", node.getName());
return map;
}
not serialized "rexList" here ^
deserialization part:
RelJsonReader#toAggCall(Map jsonAggCall) {
... cut ...
return AggregateCall.create(aggregation, distinct, false, false,
ImmutableList.of(), <-- empty list
operands,
filterOperand == null ? -1 : filterOperand,
null, RelCollations.EMPTY, type, name);
}
used ImmutableList.of() instead of "rexList" (no place where we can
obtain it here, ok)
and further call for assembly:
public static AggregateCall create(...
...
final List preTypes = RexUtil.types(rexList); <- pre
types are empty
final List types =
SqlTypeUtil.projectTypes(input.getRowType(), argList); <-
argList is empty too
final Aggregate.AggCallBinding callBinding =
new Aggregate.AggCallBinding(typeFactory, aggFunction,
preTypes,
types, groupCount, filterArg >= 0);
type = aggFunction.inferReturnType(callBinding);
private static RelDataType inferReturnType(SqlOperatorBinding
opBinding) {
// LITERAL_AGG takes one pre-operand and zero (post-)operands.
if (opBinding.getPreOperandCount() != 1
|| opBinding.getOperandCount() != 1) {
throw new AssertionError(); <-- brings this assertion
}
return opBinding.getOperandType(0);
}
it`s not clear for me how to write test for such a case.
I realize that description is probably not clear but i tried to make it
as informative as i can.
If all above is not clear, brief question : how to write
serizlization\deserialization tests for such Operands, yes i found
RelWriterTest but it still unclear for me:
@Test void testWriter1() {
int i = Frameworks.withPlanner((cluster, relOptSchema, rootSchema)
-> {
final RelDataTypeFactory typeFactory = cluster.getTypeFactory();
RelDataTypeField field = new RelDataTypeFieldImpl(
"ID", 0, typeFactory.createSqlType(SqlTypeName.INTEGER)
);
RelRecordType rowType = new
RelRecordType(Collections.singletonList(field));
RelOptAbstractTable t1 = new RelOptAbstractTable(null, "t1",
rowType) {
@Override
public T unwrap(Class clazz) {
return null;
}
};
LogicalTableScan lt = new LogicalTableScan(cluster, null,
List.of(), t1);
RelDataTypeSystem typeSys =
cluster.getTypeFactory().getTypeSystem();
SqlBasicAggFunction literalFunc = SqlLiteralAggFunction.INSTANCE;
RexLiteral rex = RexLiteral.fromJdbcString(
new BasicSqlType(typeSys, SqlTypeName.BOOLEAN),
SqlTypeName.BOOLEAN, "true");
AggregateCall type = AggregateCall.create(literalFunc, false,
false, false, List.of(rex),
ImmutableList.of(), -1, null,
RelCollations.of(new RelFieldCollation(1)), new
BasicSqlType(typeSys, SqlTypeName.BOOLEAN), "i");
LogicalAggregate la = new LogicalAggregate(cluster, null,
List.of(), lt, ImmutableBitSet.of(), null, List.of(type));
final JsonBuilder jsonBuilder = new JsonBuilder();
final RelJson json = RelJson.create().withJsonBuilder(jsonBuilder);
final Object o = json.toJson(la);
assertThat(o, notNullValue());
final String s = jsonBuilder.toJsonString(o);
//final String expectedJson = "";
//assertThat(s, is(expectedJson));
final RelDataType type2 = json.toType(typeFactory, o);
assertThat(type2, is(type));
return 0;
});
}
Thanks !
LITERAL_AGG question
hello, please can someone explain me how LITERAL_AGG need to be correctly
serialized\deserealized and how to write appropriate tests.
Problem :
we have LITERAL_AGG
further:
toJson(org.apache.calcite.rel.core.AggregateCall) {
Map map = map();
map.put("agg", toJson(node.getAggregation()));
map.put("type", toJson(node.getType()));
map.put("distinct", node.isDistinct());
map.put("operands", node.getArgList());
map.put("filter", node.filterArg);
map.put("name", node.getName());
return map;
}
not serialized "rexList" here ^
deserialization part:
RelJsonReader#toAggCall(Map jsonAggCall) {
... cut ...
return AggregateCall.create(aggregation, distinct, false, false,
ImmutableList.of(), <-- empty list
operands,
filterOperand == null ? -1 : filterOperand,
null, RelCollations.EMPTY, type, name);
}
used ImmutableList.of() instead of "rexList" (no place where we can obtain
it here, ok)
and further call for assembly:
public static AggregateCall create(...
...
final List preTypes = RexUtil.types(rexList); <- pre
types are empty
final List types =
SqlTypeUtil.projectTypes(input.getRowType(), argList); <-
argList is empty too
final Aggregate.AggCallBinding callBinding =
new Aggregate.AggCallBinding(typeFactory, aggFunction, preTypes,
types, groupCount, filterArg >= 0);
type = aggFunction.inferReturnType(callBinding);
private static RelDataType inferReturnType(SqlOperatorBinding opBinding)
{
// LITERAL_AGG takes one pre-operand and zero (post-)operands.
if (opBinding.getPreOperandCount() != 1
|| opBinding.getOperandCount() != 1) {
throw new AssertionError(); <-- brings this assertion
}
return opBinding.getOperandType(0);
}
it`s not clear for me how to write test for such a case.
I realize that description is probably not clear but i tried to make it as
informative as i can.
If all above is not clear, brief question : how to write
serizlization\deserialization tests for such Operands, yes i found
RelWriterTest but it still unclear for me:
@Test void testWriter1() {
int i = Frameworks.withPlanner((cluster, relOptSchema, rootSchema) -> {
final RelDataTypeFactory typeFactory = cluster.getTypeFactory();
RelDataTypeField field = new RelDataTypeFieldImpl(
"ID", 0, typeFactory.createSqlType(SqlTypeName.INTEGER)
);
RelRecordType rowType = new
RelRecordType(Collections.singletonList(field));
RelOptAbstractTable t1 = new RelOptAbstractTable(null, "t1",
rowType) {
@Override
public T unwrap(Class clazz) {
return null;
}
};
LogicalTableScan lt = new LogicalTableScan(cluster, null, List.of(),
t1);
RelDataTypeSystem typeSys = cluster.getTypeFactory().getTypeSystem();
SqlBasicAggFunction literalFunc = SqlLiteralAggFunction.INSTANCE;
RexLiteral rex = RexLiteral.fromJdbcString(
new BasicSqlType(typeSys, SqlTypeName.BOOLEAN),
SqlTypeName.BOOLEAN, "true");
AggregateCall type = AggregateCall.create(literalFunc, false, false,
false, List.of(rex),
ImmutableList.of(), -1, null,
RelCollations.of(new RelFieldCollation(1)), new
BasicSqlType(typeSys, SqlTypeName.BOOLEAN), "i");
LogicalAggregate la = new LogicalAggregate(cluster, null, List.of(),
lt, ImmutableBitSet.of(), null, List.of(type));
final JsonBuilder jsonBuilder = new JsonBuilder();
final RelJson json = RelJson.create().withJsonBuilder(jsonBuilder);
final Object o = json.toJson(la);
assertThat(o, notNullValue());
final String s = jsonBuilder.toJsonString(o);
//final String expectedJson = "";
//assertThat(s, is(expectedJson));
final RelDataType type2 = json.toType(typeFactory, o);
assertThat(type2, is(type));
return 0;
});
}
Thanks !
Re: [jira] [Created] (CALCITE-5950) Default column constraint is erroneously processed.
Julian thanks. PR passed all checks and ready for review. I just took a look at https://github.com/apache/calcite/pull/3393/checks. I see an autostyle violation, and it looks valid. That error is there so that you can fix it and not waste reviewers' time. On Wed, Aug 23, 2023 at 2:15 PM Julian Hyde wrote: There aren't many false positives or flaky tests in CI, checker-framework, error-prone. If it says that (say) you are passing a nullable object to a method that requires a not-nullable argument, you probably are. I find it useful to run checker-framework on my own computer, rather than waiting for CI. But you must use JDK 11: ./gradlew --no-parallel --no-daemon -PenableCheckerframework :linq4j:classes :core:classes Julian On Wed, Aug 23, 2023 at 8:42 AM stanilovsky evgeny wrote: > > PR is ready for review, plz explain did i need to fix all of: CI / > CheckerFramework issues ? > Seems some of them are false positive. > thanks ! > > > Evgeny Stanilovsky created CALCITE-5950:
Re: [jira] [Created] (CALCITE-5950) Default column constraint is erroneously processed.
PR is ready for review, plz explain did i need to fix all of: CI / CheckerFramework issues ? Seems some of them are false positive. thanks ! Evgeny Stanilovsky created CALCITE-5950:
Re: [Question] Working with defaults in calcite.
Thanks for pointing a right test, seems defaults are broken: such test is failed. issue created : https://issues.apache.org/jira/browse/CALCITE-5950 # Create a basic table DEFAULT create table tdef (i int not null, j int default 100); (0 rows modified) !update insert into tdef values (1, DEFAULT); (1 row modified) !update insert into tdef(i) values (2); (1 row modified) !update select * from tdef order by i; +---+-+ | I | J | +---+-+ | 1 | 100 | | 2 | 100 | +---+-+ (2 rows) !ok but obtain from calcite: +---+-+ | I | J | +---+-+ | 1 | | | 2 | 100 | +---+-+ If you way there are no execution tests, I wouldn’t be surprised. DDL was historically deemed out of scope, and what we didn’t implement, we couldn’t test. But now we have the ’server’ component, and there are tests such as table.iq [1]. Can you add some tests? Julian [1] https://github.com/apache/calcite/blob/main/server/src/test/resources/sql/table.iq On Aug 14, 2023, at 7:40 AM, stanilovsky evgeny wrote: Hello community ! I just try a simple case, like : create table foo (i int not null, j int default 100); and obtain parser error: parse failed: Encountered "default" at line 1, column 41. while create table foo (i int not null, j int); insert into foo values (1, DEFAULT); throws no exceptions in quidem test framework i also find SqlValidatorTest#testInsertShouldNotCheckForDefaultValue that contains newColumnDefaultValue: // Now remove DEPTNO, which has a default value, from the target list. // Will generate an extra call to newColumnDefaultValue at sql-to-rel time, // just not yet. final String sql4 = "insert into ^emp^ (empno, ename, job, mgr, hiredate,\n" + " sal, comm, slacker)\n" + "values(1, 'nom', 'job', 0,\n" + " timestamp '1970-01-01 00:00:00', 1, 1, false)"; org.apache.calcite.sql2rel.InitializerExpressionFactory#newColumnDefaultValue so is it correct that: 1. calcite has no execution tests for CREATE TABLE with DEFAULT columns ? 2. seems can`t process correctly such kind of columns with insertions ? (i see no tests) CREATE TABLE integers(i INTEGER PRIMARY KEY, col1 INTEGER DEFAULT 200, col2 INTEGER DEFAULT 100) INSERT INTO integers VALUES (1, DEFAULT, DEFAULT) INSERT INTO integers(i, col2) VALUES (2, DEFAULT), (3, 4), (4, DEFAULT)"); thanks !
[Question] Working with defaults in calcite.
Hello community ! I just try a simple case, like : create table foo (i int not null, j int default 100); and obtain parser error: parse failed: Encountered "default" at line 1, column 41. while create table foo (i int not null, j int); insert into foo values (1, DEFAULT); throws no exceptions in quidem test framework i also find SqlValidatorTest#testInsertShouldNotCheckForDefaultValue that contains newColumnDefaultValue: // Now remove DEPTNO, which has a default value, from the target list. // Will generate an extra call to newColumnDefaultValue at sql-to-rel time, // just not yet. final String sql4 = "insert into ^emp^ (empno, ename, job, mgr, hiredate,\n" + " sal, comm, slacker)\n" + "values(1, 'nom', 'job', 0,\n" + " timestamp '1970-01-01 00:00:00', 1, 1, false)"; org.apache.calcite.sql2rel.InitializerExpressionFactory#newColumnDefaultValue so is it correct that: 1. calcite has no execution tests for CREATE TABLE with DEFAULT columns ? 2. seems can`t process correctly such kind of columns with insertions ? (i see no tests) CREATE TABLE integers(i INTEGER PRIMARY KEY, col1 INTEGER DEFAULT 200, col2 INTEGER DEFAULT 100) INSERT INTO integers VALUES (1, DEFAULT, DEFAULT) INSERT INTO integers(i, col2) VALUES (2, DEFAULT), (3, 4), (4, DEFAULT)"); thanks !
Re: DECIMAL(2, 3) meaning
Stamatis, as i can see this discussion is about values\literals validation
instead of NUMERIC types, not only types at all.
I would say that type checks for precision/scale etc would fit better
inside the RelDataTypeFactory and its respective type system.
Best,
Stamatis
On Mon, Aug 7, 2023 at 12:39 AM wrote:
I found this documentation for Oracle DECIMAL data type:
https://docs.oracle.com/en/database/oracle/oracle-database/23/sqlrf/Data-Types.html#GUID-75209AF6-476D-4C44-A5DC-5FA70D701B78,
which explains what a SCALE > PRECISION should mean.
> Scale can be greater than precision, most commonly when e notation is
used. When scale is greater than precision, the precision specifies the
maximum number of significant digits to the right of the decimal point.
For example, a column defined as NUMBER(4,5) requires a zero for the
first digit after the decimal point and rounds all values past the
fifth digit after the decimal point.
Let me ask a related question: in my backend I want to reject such
numbers. What is the right way to do it?
Should this be done in a SqlShuttle? Or should some Validator class be
extended?
Thank you,
Mihai
-Original Message-
From: Julian Hyde
Sent: Sunday, August 06, 2023 2:19 PM
To: dev@calcite.apache.org
Subject: Re: DECIMAL(2, 3) meaning
As I commented in https://issues.apache.org/jira/browse/CALCITE-5901, I
don’t think it’s a bug to support behavior beyond what the standard
requires. Which Calcite does, intentionally.
Julian
> On Aug 6, 2023, at 08:35, stanilovsky evgeny
wrote:
>
> Ok, seems like a bug.
> Feel free to fill the issue.
>
>> I have added this test to SqlOperatorTest:
>>
>>f.checkScalar("cast(0.012 as DECIMAL(2, 5))", new
BigDecimal("0.012"),
>>"DECIMAL(2, 5) NOT NULL");
>>
>> and it has passed. That's why I am asking. It should fail, but it
doesn't.
>>
>> Mihai
>>
>> -Original Message-
>> From: stanilovsky evgeny
>> Sent: Friday, August 04, 2023 7:00 AM
>> To: dev@calcite.apache.org
>> Subject: Re: DECIMAL(2, 3) meaning
>>
>> Hello Mihai.
>> A bit older standard describes Precision as : Precision of decimal
floating-point values is a positive value that specifies the number of
significant decimal digits in the mantissa.
>>
>> Thus:
>> cast(0.012 as DECIMAL(3, 3)) - ok
>> cast(0.012 as DECIMAL(2, 3)) - fail
>> cast(0.012 as DECIMAL(1, 3)) - fail
>> cast(0.012 as DECIMAL(2, 5)) - fail
>>
>>
>>> Hello,
>>>
>>>
>>> I notice that Calcite happily accepts decimal type specifications
>>> where the scale is greater than the precision.
>>>
>>> There are quite a few tests with such types.
>>>
>>>
>>> What is the meaning of such types?
>>>
>>>
>>> The SQL 92 standard has this statement on page 109:
>>>
>>>
>>> 15)The of an shall not be greater than
>>>
>>>the of the .
>>>
>>>
>>> Thank you,
>>>
>>> Mihai
Re: DECIMAL(2, 3) meaning
Ok, seems like a bug.
Feel free to fill the issue.
I have added this test to SqlOperatorTest:
f.checkScalar("cast(0.012 as DECIMAL(2, 5))", new
BigDecimal("0.012"),
"DECIMAL(2, 5) NOT NULL");
and it has passed. That's why I am asking. It should fail, but it
doesn't.
Mihai
-----Original Message-
From: stanilovsky evgeny
Sent: Friday, August 04, 2023 7:00 AM
To: dev@calcite.apache.org
Subject: Re: DECIMAL(2, 3) meaning
Hello Mihai.
A bit older standard describes Precision as : Precision of decimal
floating-point values is a positive value that specifies the number of
significant decimal digits in the mantissa.
Thus:
cast(0.012 as DECIMAL(3, 3)) - ok
cast(0.012 as DECIMAL(2, 3)) - fail
cast(0.012 as DECIMAL(1, 3)) - fail
cast(0.012 as DECIMAL(2, 5)) - fail
Hello,
I notice that Calcite happily accepts decimal type specifications
where the scale is greater than the precision.
There are quite a few tests with such types.
What is the meaning of such types?
The SQL 92 standard has this statement on page 109:
15)The of an shall not be greater than
the of the .
Thank you,
Mihai
Re: DECIMAL(2, 3) meaning
Hello Mihai. A bit older standard describes Precision as : Precision of decimal floating-point values is a positive value that specifies the number of significant decimal digits in the mantissa. Thus: cast(0.012 as DECIMAL(3, 3)) - ok cast(0.012 as DECIMAL(2, 3)) - fail cast(0.012 as DECIMAL(1, 3)) - fail cast(0.012 as DECIMAL(2, 5)) - fail Hello, I notice that Calcite happily accepts decimal type specifications where the scale is greater than the precision. There are quite a few tests with such types. What is the meaning of such types? The SQL 92 standard has this statement on page 109: 15)The of an shall not be greater than the of the . Thank you, Mihai
Re: [Question] CAST possibility cases
Thanks a lot for response. Ok i found partial CAST tests like : SqlOperatorTest#testCastBooleanToNumeric, SqlOperatorTest#testCastToString and so on. Soon i will try to fix all observed CAST discrepancy according to: ISO/IEC 9075-2:1999 6.22 standard Hi,stanilovsky, CALCITE-5662 <https://issues.apache.org/jira/browse/CALCITE-5662> CAST(BOOLEAN as INTEGER) throws a NumberFormatException. Maybe you can get some information from this JIRA. Best, LakeShen Julian Hyde 于2023年8月2日周三 23:04写道： I think there’s at least one jira case discussing Boolean cast. Can you locate it? > On Aug 2, 2023, at 5:46 AM, stanilovsky evgeny < estanilovs...@gridgain.com> wrote: > > hello all ! > According to sql standard, chapter: 6.22 > We can found, for example, that casting from BOOLEAN is deprecated into Exact Numeric, but according to SqlTypeCoercionRule > it`s possible for now. Probably i miss something, can someone highlight me in such a case? If no - did i need to fill the appropriate issue ? > Also i can`t found sufficient CAST tests ( > > thanks !
[Question] CAST possibility cases
hello all ! According to sql standard, chapter: 6.22 We can found, for example, that casting from BOOLEAN is deprecated into Exact Numeric, but according to SqlTypeCoercionRule it`s possible for now. Probably i miss something, can someone highlight me in such a case? If no - did i need to fill the appropriate issue ? Also i can`t found sufficient CAST tests ( thanks !
Re: Easier and more comprehensive testing
Hello, your github link doesn`t open.
plz check discussion here in dev list titled:
Running Sql Logic Tests for Calcite
This is the JIRA case: https://issues.apache.org/jira/browse/CALCITE-5615
And this is the PR: https://github.com/apache/calcite/pull/3145
Hello,
I am working to test our calcite-based compiler
(https://github.com/feldera/dbsp/sql-to-dbsp-compiler), and since I am
lazy
I am borrowing tests from other open-source test suites, like Postgres.
I am finding bugs in Calcite with a relatively high frequency, as you may
have noticed if you follow the JIRA. I would say that one in 3 SQL
functions
I test turns out to have some problems. So our techniques seem to be
effective at finding bugs.
I think that some of the techniques we are using could be applied to
Calcite
as well:
* First, we run all tests with and without the optimizer. In
particular, we go through all the "constant evaluation" rules of Calcite.
For constant expressions the results should be identical with and without
optimizer. They aren't always, and we have found quite a few cases where
the
compile-time evaluation crashes or produces wrong results. So the
compile-time and the run-time evaluator cross-check each other.
* Second, we have tried to make it very easy to write end-to-end
tests, at least positive tests (which are supposed to return a value
rather
than an error). Here is a stylized excerpt from our testing code:
@Override
public void prepareData() {
this.executeStatements("CREATE TABLE FLOAT4_TBL (f1 float4);\n"
+
"INSERT INTO FLOAT4_TBL(f1) VALUES ('0.0');\n" +
"INSERT INTO FLOAT4_TBL(f1) VALUES ('1004.30 ');\n" +
"INSERT INTO FLOAT4_TBL(f1) VALUES (' -34.84
');\n" +
"INSERT INTO FLOAT4_TBL(f1) VALUES
('1.2345678901234e+20');\n" +
"INSERT INTO FLOAT4_TBL(f1) VALUES
('1.2345678901234e-20');");
}
@Test
public void testFPArithmetic() {
this.qs("SELECT f.f1, f.f1 * '-10' AS x FROM FLOAT4_TBL f\n" +
" WHERE f.f1 > '0.0';\n" +
" f1 | x\n" +
"---+\n" +
"1004.3 | -10043\n" +
" 1.2345679e+20 | -1.2345678e+21\n" +
" 1.2345679e-20 | -1.2345678e-19\n" +
"(3 rows)\n" +
"\n" +
"SELECT f.f1, f.f1 + '-10' AS x FROM FLOAT4_TBL f\n" +
" WHERE f.f1 > '0.0';\n" +
" f1 | x \n" +
"---+---\n" +
"1004.3 | 994.3\n" +
" 1.2345679e+20 | 1.2345679e+20\n" +
" 1.2345679e-20 | -10\n" +
"(3 rows)\n" +
"\n");
}
These long strings are an almost direct copy-and-paste from tests in
https://github.com/postgres/postgres/blob/03734a7fed7d924679770adb78a7db8a37
d14188/src/test/regress/expected/float4.out (but they have to manually
checked, since Calcite does often not match Postgres behaviors). A string
can contain multiple query-expected result strings.
The point is that the barrier for writing the tests is quite low.
Granted, this approach has some weaknesses as well, in particular, we
rely
on a specific output format and is brittle in some respects (e.g.,
spaces in
output strings).
I have seen some *.iq files as resources in the Calcite source code, are
these used in the same way for testing in Calcite? If yes, that's great,
and there should be many more.
Calcite has lots of unit tests, but I find that our "end-to-end" tests
are
easier to write and have much better code coverage. For example, for each
failure I find in Calcite I have to dig pretty hard to figure out which
of
the testing files should contain my reproduction (e.g., SqlOperatorTest,
RelOptRulesTest, RelToSqlConverterTest, etc.), and how to exactly write
the
reproduction (some reproductions require editing some huge XML files
too).
That takes a lot of time.
I am not necessarily signing up to build this infrastructure, at least
not
right away. I am not sure how much of the stuff I wrote for our compiler
could be ported directly to Calcite. But I will think about it, and I
will
gladly help this effort.
I appreciate any comments and suggestions,
Mihai
Re: Force push to calcite main
I already told that community need to vote for prohibit force push. Hello, It appears that there was a force push to main yesterday [1] rewriting the history for a bunch of commits. I don't know if it was intentional or not but it seems that now resolved JIRAs (after CALCITE-5810 I think) are pointing to non-existent commits. Can someone please update the JIRA tickets with the correct commit hashes and also ensure that we didn't lose anything after the rebase? Best, Stamatis [1] https://lists.apache.org/thread/7jjnbkkh9tv49sjcc5kg2tm7c54tj861
Re: [DISCUSS] Towards Calcite 1.35.0
[1] Is ready for merging, approvals are obtained. [1] https://github.com/apache/calcite/pull/3211 I finished CALCITE-5701 Add NAMED_STRUCT function (enabled in Spark library) I hope it's not too late for the release: https://github.com/apache/calcite/pull/3295 The CI passes, it just needs a re-run on the last commit. On Fri, Jun 30, 2023 at 5:26 PM Gian Merlino wrote: Of the open tickets, I reviewed CALCITE-5708 and CALCITE-5727, which are two I felt I understood well enough to review. On 2023/06/28 19:36:16 Julian Hyde wrote: > I have taken care of 1, 2, 3, 4. > > Tanner, can you do 5. > > Of the 12 open issues that are open and have fixVersion = 1.35, most have PRs ready for review [1]. Can a few committers each take say 3 cases each and review their PRs? > > Julian > > [1] https://issues.apache.org/jira/issues/?jql=project%20%3D%20CALCITE%20AND%20fixVersion%20%3D%201.35.0%20and%20status%20%3D%20open > > > On Jun 28, 2023, at 6:25 AM, Ruben Q L wrote: > > > > Hello, > > > > Thanks for checking Xiong Duan, I have taken care of 6,7,8. > > > > Contributors, please remember that after completing a ticket, the Jira > > needs to be set to "Resolved", not "Closed" (they will be moved to Closed > > by the Release Manager, once the next release is produced). > > > > Apart from that, our dashboard [1] still shows 22 unresolved tickets with > > fixVersion=1.35 > > Except from the blocker ones, I guess the rest would need to be resolved > > shortly or otherwise moved to fixVersion=1.36 (or no fixVersion). > > As a general rule, please do not set a fixVersion unless it is a blocker > > issue, or you're reasonably sure that the issue will be done for the next > > release. > > > > Best, > > Ruben > > > > [1] > > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950 > > > > > > > > > > On Wed, Jun 28, 2023 at 1:47 PM xiong duan wrote: > > > >> In order to release 1.35.0, I reviewed all the fixed issues in this > >> version, Here are some issue statuses that need to be resolved: > >> > >> > >> 1. https://issues.apache.org/jira/browse/CALCITE-5764(The PR has > >> merge.But issue is IN PROGRESS). (Assigine Julian Hyde) > >> 2. https://issues.apache.org/jira/browse/CALCITE-5706(The PR has > >> merge.But issue is IN PROGRESS). (Assigine Julian Hyde) > >> 3. https://issues.apache.org/jira/browse/CALCITE-5765(The PR has > >> merge.But issue is IN PROGRESS). (Assigine Julian Hyde) > >> 4. https://issues.apache.org/jira/browse/CALCITE-5762(The PR has > >> merge.But issue is IN PROGRESS). (Assigine Julian Hyde) > >> 5. https://issues.apache.org/jira/browse/CALCITE-5747(The PR has > >> merge.But issue is NOT Resolve). (Assigine Tanner Clary) > >> 6. https://issues.apache.org/jira/browse/CALCITE-5771(The Fix Version > >> is > >> NONE, Need set it to 1.35.0) > >> 7. https://issues.apache.org/jira/browse/CALCITE-5757(The Fix Version > >> is > >> NONE, Need set it to 1.35.0) > >> 8. https://issues.apache.org/jira/browse/CALCITE-4679(Resolution should > >> be Fixed. But is Resolved) > >> > >> If you are busy, Please tell me the real status of the ISSUE, and I will > >> handle it. (6,7,8 need one PMC to handle it).Thanks. > >> > >> > >> Jacky Lau 于2023年6月26日周一 14:24写道： > >> > >>> hi @xiong duan: > >>>thanks for your review and merged very much, and i forgot there are > >>> also have one pr https://github.com/apache/calcite/pull/3262 > >>>do you also have time to have a look？ > >>> > >>> > >>> xiong duan 于2023年6月25日周日 19:45写道： > >>> > I have reviewed: > https://github.com/apache/calcite/pull/3238 > https://github.com/apache/calcite/pull/3263 > If no other problem, I will merge it tomorrow. > > Ruben Q L 于2023年6月25日周日 17:20写道： > > > I'd like to include CALCITE-5789 in 1.35. I'll try to finalize it > > today/tomorrow. > > > > > > > > El dom, 25 jun 2023, 6:33, Jacky Lau > >> escribió: > > > >> could we review this pr, so that it could be merged in 1.35. then > >> the > > 1.35 > >> will almost have the full spark collection function. > >> I would very appreciate a review. if someone has time. > >> > >> https://github.com/apache/calcite/pull/3238 > >> https://github.com/apache/calcite/pull/3263 > >> > >> xiong duan 于2023年6月25日周日 10:31写道： > >> > >>> I have created CALCITE-5797 to release 1.35.0. If there are no > >> objections, > >>> I will create an RC in the following days. > >>> > >>> [1]:https://issues.apache.org/jira/browse/CALCITE-5797 > >>> > >>> If there are other must fix for 1.35.0 please let us know so that > >>> we > > can > >>> plan accordingly. > >>> > >>> Julian Hyde 于2023年6月24日周六 01:58写道： > >>> > We said ‘mid-June’. Time to get this release rolling? > > > On Jun 20, 2023, at 10:34 PM, Gian Merlino > > wrote: > > > > I committed
Please make a review pr 3211
There is ones more change touching null literals in SUBSTRING func. Can someone make a review plz? https://issues.apache.org/jira/browse/CALCITE-5708 https://github.com/apache/calcite/pull/3211
Re: RelDataTypeSystemImpl#deriveSumType question.
Thanks, ok standard contract it as __implementation-defined__, now it`s clear. Hi stanilovsky, You can customize your behavior in RelDataTypeSystem. We have discussed this issue before, you could find more details in [1] and [2]. - [1] https://issues.apache.org/jira/browse/CALCITE-5604 - [2] https://issues.apache.org/jira/browse/CALCITE-1945 Best, Dan Zou 2023年4月25日 19:14，stanilovsky evgeny 写道： Hello ! Probably can anyone explain why RelDataTypeSystemImpl#deriveSumType returns INTEGER for incoming INTEGER type ? I run simple: SELECT i, SUM(i) FROM integers i1 GROUP BY i ORDER BY i; and i expect that return type will be BIGINT. Did i miss something here ? Thanks !
RelDataTypeSystemImpl#deriveSumType question.
Hello ! Probably can anyone explain why RelDataTypeSystemImpl#deriveSumType returns INTEGER for incoming INTEGER type ? I run simple: SELECT i, SUM(i) FROM integers i1 GROUP BY i ORDER BY i; and i expect that return type will be BIGINT. Did i miss something here ? Thanks !
Re: Insert data exceeding fixed length column.
I rewrite test with end-to-end usage (hope it`s correct) and put it into
blank.iq (test is near) and found that it not follow the sql standard.
Also probably someone can suggest how can i write test with dynamic params
involved ? I found that Quidem has no appropriate functionality.
Thanks !
create table fix_char_col (i int, c char(5));
(0 rows modified)
!update
insert into fix_char_col values (1, 'a'), (2, 'aa'), (3, '
aa'), (4, 'aa'), (5, ' aa');
(1 row modified)
!update
SELECT * FROM fix_char_col ORDER BY i;
+---+---+
| I | C |
+---+---+
| 1 | a |
| 2 | a |
| 3 | |
| 4 | aa|
| 5 | aa |
+---+---+
(2 rows)
!ok
and as a result :
expected:
< +---+---+
< | I | C |
< +---+---+
< | 1 | a |
< | 2 | a |
< | 3 | |
< | 4 | aa|
< | 5 | aa |
< +---+---+
calcite output:
+---+-+
| I | C |
+---+-+
| 1 | a |
| 2 | aa |
| 3 | aa |
| 4 | aa |
| 5 | aa |
+---+-+
Ok, thanks i understand what you are talking about, i will move the test
and fill issue if it necessary.
I agree that that query should throw when executed. If it doesn’t, you
should log a bug.
However, that test in SqlToRelConverterTest doesn’t prove that Calcite
doesn’t throw at execute time.
On Feb 8, 2023, at 11:20 PM, stanilovsky evgeny
wrote:
Hello, seems sql standard regulate situation when insertion (char,
varchar) data exceeds fixed len column as :
If the declared type T is fixed length string with length in
characters L and the
length in characters M of V is larger than L, then
Case:
a) If the right part M–L characters of V are all space`s, then the
value of T is set to the first
L characters of V.
b) If one or more of the right part M–L characters of V are not
space`s, then exception is raised.
But fast test [1] in SqlToRelConverterTest highlights that calcite not
satisfy of such a rule.
I miss something or need to fill the issue ?
Thanks !
[1]
@Test void testInsert1() {
String moreThanVarcharLimit = "a".repeat(30);
final String sql = "insert into emp (empno, ename, job, mgr,
hiredate,\n"
+ " sal, comm, deptno, slacker)\n"
+ "values(1, '" + moreThanVarcharLimit + "', 'job', 0,\n"
+ " timestamp '1970-01-01 00:00:00', 1, 1, 1, false)";
sql(sql).ok();
}
Re: Insert data exceeding fixed length column.
Ok, thanks i understand what you are talking about, i will move the test
and fill issue if it necessary.
I agree that that query should throw when executed. If it doesn’t, you
should log a bug.
However, that test in SqlToRelConverterTest doesn’t prove that Calcite
doesn’t throw at execute time.
On Feb 8, 2023, at 11:20 PM, stanilovsky evgeny
wrote:
Hello, seems sql standard regulate situation when insertion (char,
varchar) data exceeds fixed len column as :
If the declared type T is fixed length string with length in characters
L and the
length in characters M of V is larger than L, then
Case:
a) If the right part M–L characters of V are all space`s, then the
value of T is set to the first
L characters of V.
b) If one or more of the right part M–L characters of V are not
space`s, then exception is raised.
But fast test [1] in SqlToRelConverterTest highlights that calcite not
satisfy of such a rule.
I miss something or need to fill the issue ?
Thanks !
[1]
@Test void testInsert1() {
String moreThanVarcharLimit = "a".repeat(30);
final String sql = "insert into emp (empno, ename, job, mgr,
hiredate,\n"
+ " sal, comm, deptno, slacker)\n"
+ "values(1, '" + moreThanVarcharLimit + "', 'job', 0,\n"
+ " timestamp '1970-01-01 00:00:00', 1, 1, 1, false)";
sql(sql).ok();
}
Insert data exceeding fixed length column.
Hello, seems sql standard regulate situation when insertion (char,
varchar) data exceeds fixed len column as :
If the declared type T is fixed length string with length in characters L
and the
length in characters M of V is larger than L, then
Case:
a) If the right part M–L characters of V are all space`s, then the value
of T is set to the first
L characters of V.
b) If one or more of the right part M–L characters of V are not space`s,
then exception is raised.
But fast test [1] in SqlToRelConverterTest highlights that calcite not
satisfy of such a rule.
I miss something or need to fill the issue ?
Thanks !
[1]
@Test void testInsert1() {
String moreThanVarcharLimit = "a".repeat(30);
final String sql = "insert into emp (empno, ename, job, mgr,
hiredate,\n"
+ " sal, comm, deptno, slacker)\n"
+ "values(1, '" + moreThanVarcharLimit + "', 'job', 0,\n"
+ " timestamp '1970-01-01 00:00:00', 1, 1, 1, false)";
sql(sql).ok();
}
Re: [ANNOUNCE] New committer: Alex Plehanov
Congratulations, Alex! Welcome, Alex! Andrei Sereda 于2023年1月11日周三 01:16写道： Welcome, Alex! On Tue, Jan 10, 2023, 11:01 Stamatis Zampetakis wrote: > Welcome Alex! Looking forward to working with you. > > Best, > Stamatis > > On Tue, Jan 10, 2023 at 2:48 AM Chunwei Lei > wrote: > > > Welcome, Alex! > > > > > > Best, > > Chunwei > > > > > > On Sun, Jan 8, 2023 at 3:59 PM Alex Plehanov > > wrote: > > > > > Hello, > > > > > > Thanks for the warm welcome! A couple of words about myself: In Apache > > > Ignite we are developing the new SQL engine (based on Apache Calcite) > as > > a > > > replacement for the old SQL engine. The old engine has some > > > fundamental limitations which we can resolve with the help of Calcite. > I > > am > > > one of the major contributors to this part of the Apache Ignite. The > new > > > SQL engine was first released with Ignite 2.13 (2022-04-26) but still > in > > > beta status. Our primary goal now - make it production ready. > > > > > > сб, 7 янв. 2023 г. в 08:45, Benchao Li : > > > > > > > Congratulations, Alex! > > > > > > > > Francis Chuang 于2023年1月7日周六 09:51写道： > > > > > > > > > Congrats, Alex! > > > > > > > > > > On 7/01/2023 5:17 am, Michael Mior wrote: > > > > > > Welcome Alex! > > > > > > > > > > > > -- > > > > > > Michael Mior > > > > > > mm...@apache.org > > > > > > > > > > > > > > > > > > On Fri, Jan 6, 2023 at 6:04 AM Ruben Q L > > wrote: > > > > > > > > > > > >> Apache Calcite's Project Management Committee (PMC) has invited > > Alex > > > > > >> Plehanov to become a committer, and we are pleased to announce > > that > > > he > > > > > has > > > > > >> accepted the invitation. > > > > > >> > > > > > >> Alex is an Apache Ignite PMC member and the main author of the > > > > > >> ignite-calcite module. He has done significant contributions on > > > > Calcite, > > > > > >> especially fixing bugs detected by Ignite. > > > > > >> > > > > > >> Alex, welcome, thank you for your contributions, and we look > > forward > > > > to > > > > > >> your further interactions with the community! If you wish, > please > > > feel > > > > > free > > > > > >> to tell us a bit more about yourself and what you are currently > > > > working > > > > > on. > > > > > >> As your first commit, you can add yourself to the contributors > > list > > > > [1] > > > > > and > > > > > >> the community page will re-generate [2]. > > > > > >> > > > > > >> Ruben (on behalf of the Apache Calcite PMC) > > > > > >> > > > > > >> [1] > > > > > >> > > > > > > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml > > > > > >> [2] https://calcite.apache.org/community/#project-members > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Best, > > > > Benchao Li > > > > > > > > > >
[REVIEW REQUEST] CALCITE-5253 NATURAL JOIN erroneous validation.
Hi, I would like to request a review for CALCITE-5253, pr [1], it fixes partially erroneous natural join validaton logic. Thanks ! [1] https://github.com/apache/calcite/pull/2889 [2] https://issues.apache.org/jira/browse/CALCITE-5253
Re: Broken history after force pushing to master
Just notice - github allows to disable a force push into master branch. It's not the first time that we have had small problems with history so no worries. Thankfully with the help of commit@calcite list we can always find a way to fix things as long as we identify the problem soon enough. According to the change log [1] the last commit before force pushing was (dcbc493), which corresponds to CALCITE-5019. * -- * -- B -- O -- O -- O (dcbc493) \ N -- N -- N refs/heads/master (c3dbf52) According to [2] the full commit id is dcbc493bf699d961427952c5efc047b76d859096. In order to restore the master branch in the state that it was before the force-push (before release) I plan to do the following steps: git fetch origin dcbc493bf699d961427952c5efc047b76d859096 git checkout dcbc493bf699d961427952c5efc047b76d859096 git branch -D master git switch -c master git push origin master -f I will apply the above sequence in 12h from now to give some time to others to react if necessary. Obviously this will nuke out any current release candidate so we will need to cancel existing votes and create an RC2. There has been a force push also to the site branch but doesn't matter much since we can force push master to site after the release is finalized. Best, Stamatis [1] https://lists.apache.org/thread/gkvn5hlmm3jlcklgw9k9nodyhxvqmsw4 [2] https://lists.apache.org/thread/rvngk5tygfoyoc0klhwpo717mrngkdrw On Wed, Mar 9, 2022 at 6:44 PM Ruben Q L wrote: Hello Liya, No worries, we all make mistakes. I think the sequence of steps that you describe looks like a plausible explanation for how we get into this situation. Do you know (from step 2) which commits were in site branch that were not in master? If in the future you (or anybody else) get blocked or experience any problem on a certain step during the release process, do not hesitate to send an email to the dev list with subject "[HELP] ..." describing the issue. In my experience, someone from the community will assist relatively fast. Any git expert with a clear idea on how to restore the master branch? Best, Ruben On Wed, Mar 9, 2022 at 1:32 PM Fan Liya wrote: > Hi all, > > I think the broken history was caused by this: > > 1. In document "Making a release candidate [1]", it says "Make sure master > branch and site branch are in sync". > 2. I checked the two branches, and find they have diverged. Some commits in > the site branch are not in the master branch. > 3. I tried the method given in the document "git reset --hard site", but it > didn't work. > 3. I tried to cherry-pick the commits to master, but it required resolving > conflicts, because the committing order was not correct. > 4. So I used "git rebase -i" to insert the commits into the "right" place > of the master branch. > 5. Finally, I pushed the result to the original master branch. > > I think that is the reason for the broken history. Really sorry for the > trouble. > If needed, I can restore the original master branch. I have backed up the > branch. > > Best, > Liya Fan > > [1] https://calcite.apache.org/docs/howto.html#making-a-release-candidate > > xiong duan 于2022年3月9日周三 19:35写道： > > > Hi. Stamatis. I agree we need to address this issue first. > > I find some relative descriptions at end of the email > > https://lists.apache.org/thread/gkvn5hlmm3jlcklgw9k9nodyhxvqmsw4. So it > is > > a force push. Sorry I am not very good at Github job flow. But I think it > > describes what happened according to the appearances. So I hope this can > > help. > > > > This update added new revisions after undoing existing revisions. That is > > to say, some revisions that were in the old version of the branch are not > > in the new version. This situation occurs when a user --force pushes a > > change and generates a repository containing something like this: * -- * > -- > > B -- O -- O -- O (dcbc493) \ N -- N -- N refs/heads/master (c3dbf52) You > > should already have received notification emails for all of the O > > revisions, and so the following emails describe only the N revisions from > > the common base, B. Any revisions marked "omit" are not gone; other > > references still refer to them. Any revisions marked "discard" are gone > > forever. The 41 revisions listed above as "new" are entirely new to this > > repository and will be described in separate emails. The revisions listed > > as "add" were already present in the repository and have only been added > to > > this reference. > > > > Stamatis Zampetakis 于2022年3月9日周三 18:08写道： > > > > > Hi all, > > > > > > Something happened during the generation of the 1.30.0 release > candidate > > > and the git history is somewhat broken. > > > > > > If you use the GitHub repo and you try to pull (DON'T DO IT NOW) > changes > > > from master to update your local copy you will see that a merge commit > is > > > necessary which is not normal. > > > > > > Moreover, if you check the JIRAs resolved in
Re: calcite multi-threading problem
for example on a current calcite branch: grep janino gradle.properties janino.version=3.1.6 yes, we use drools in the same project, and drools uses janino too, is there a version list of janino which calcite supports? ---Original--- From: "Dmitry Sysolyatin"
Re: CALCITE-4833 review request.
Hi calciters ! Look [1] introduces "Support SELECT without FROM syntax" such syntax can be used with join queries, i.e. SELECT t1.empid FROM emps t1 LEFT JOIN emps t2 ON (SELECT t2.empid)<=100 Can anyone review fix or reject it, or reject this lexical construction at all, thus i can close the ticket with clear soul ) thanks ! [1] https://issues.apache.org/jira/browse/CALCITE-1120 [2] https://issues.apache.org/jira/browse/CALCITE-4833 fixed: gc pressure part still need to additional time, revert this part, all other waiting for review. Guys i found that some kinds of nested subqueries are failed [1]. i.e. SELECT t1.empid FROM emps t1 LEFT JOIN emps t2 ON (SELECT t2.empid)<=100 Fix a little bit hack as for me, but all tests are done. Additionally i fix some redundant gc pressure a bit. Plz make a review. thanks ! [1] https://issues.apache.org/jira/browse/CALCITE-4833
Re: CALCITE-4833 review request.
fixed: gc pressure part still need to additional time, revert this part, all other waiting for review. Guys i found that some kinds of nested subqueries are failed [1]. i.e. SELECT t1.empid FROM emps t1 LEFT JOIN emps t2 ON (SELECT t2.empid)<=100 Fix a little bit hack as for me, but all tests are done. Additionally i fix some redundant gc pressure a bit. Plz make a review. thanks ! [1] https://issues.apache.org/jira/browse/CALCITE-4833
CALCITE-4833 review request.
Guys i found that some kinds of nested subqueries are failed [1]. i.e. SELECT t1.empid FROM emps t1 LEFT JOIN emps t2 ON (SELECT t2.empid)<=100 Fix a little bit hack as for me, but all tests are done. Additionally i fix some redundant gc pressure a bit. Plz make a review. thanks ! [1] https://issues.apache.org/jira/browse/CALCITE-4833
Re: Can`t build master of calcite.
Thanks Stamatis, seems it`s my case too)
Does no other build it locally for such a time ?:)
I think you hit CALCITE-4798 [1] but not sure why others didn't now bump
into this so far.
There is a PR ready, I will test if it resolves the problem for me and
merge it today.
Best,
Stamatis
[1] https://issues.apache.org/jira/browse/CALCITE-4798
On Wed, Oct 6, 2021 at 9:31 AM stanilovsky evgeny <
estanilovs...@gridgain.com> wrote:
Thanks, i suppose it`s all ok with java
java -version
openjdk version "13.0.7" 2021-04-20
OpenJDK Runtime Environment (build 13.0.7+5-Ubuntu-0ubuntu120.04)
OpenJDK 64-Bit Server VM (build 13.0.7+5-Ubuntu-0ubuntu120.04, mixed
mode)
and not ok with:
build.gradle.kts
'''
val werror by props(true)
...
if (werror) {
options.compilerArgs.add("-Werror")
}
'''
> What is the exact Java version you are using?
>
> It looks like https://bugs.openjdk.java.net/browse/JDK-8032211 , and
it
> might be that newer OpenJDK include the fix.
>
> Vladimir
Re: Can`t build master of calcite.
Thanks, i suppose it`s all ok with java
java -version
openjdk version "13.0.7" 2021-04-20
OpenJDK Runtime Environment (build 13.0.7+5-Ubuntu-0ubuntu120.04)
OpenJDK 64-Bit Server VM (build 13.0.7+5-Ubuntu-0ubuntu120.04, mixed mode)
and not ok with:
build.gradle.kts
'''
val werror by props(true)
...
if (werror) {
options.compilerArgs.add("-Werror")
}
'''
What is the exact Java version you are using?
It looks like https://bugs.openjdk.java.net/browse/JDK-8032211 , and it
might be that newer OpenJDK include the fix.
Vladimir
Can`t build master of calcite.
hi ! can`t build top of calcite (sha: commit 74e97780add051cb71a122075e5bcbceb) $ ./gradlew build Task :core:compileJava /home/zstan/work/repo/calcite/core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java:39: warning: [deprecation] org.apache.calcite.rel.metadata.CachingRelMetadataProvider in org.apache.calcite.rel.metadata has been deprecated import org.apache.calcite.rel.metadata.CachingRelMetadataProvider; ^ error: warnings found and -Werror specified 1 error 1 warning do we need issue here?
Re: COALESCE returns maximum of available CHAR capacity.
I also check sqlite and mssql 2017 and obtain the same results, but i also
found that standard has differ treatment:
9.3 Set operation result data types
Function
Specify the Syntax Rules and result data types for s and s having set operators.
... skip ...
i) If any of the data types in DTS is variable-length char-
acter string, then the result data type is variable-length
character string with maximum length in characters equal
to the maximum of the lengths in characters and maximum
lengths in characters of the data types in DTS.
ii) Otherwise, the result data type is fixed-length character
string with length in characters equal to the maximum of
the lengths in characters of the data types in DTS.
But i suppose that exact sizing (i.e. 'a' instead of 'a ') is more usable,
just imagine if someone calls :
1. Form COALESCE from table1.
2. INSERT INTO SELECT Syntax with COALESCE into table2
3. Further search in table1 using table2 predicates.
'a ' not belong to table1 and cant be found, i think this is not a single
case.
community ?
Julian, thanks !
mysql differs here:
mysql> SELECT LENGTH(a) from (SELECT COALESCE('a', '') as a) as t;
+---+
| LENGTH(a) |
+---+
| 1 |
+---+
mysql> SELECT LENGTH(a) from (SELECT COALESCE('aa', '') as a) as t;
+---+
| LENGTH(a) |
+---+
| 2 |
+---+
I don`t claim that calcite need to be the same, i just want to
understand what is correct.
My point of view - if COALESCE transforms into CASE-WHEN statement and
returns first not null it strange instead of expected 'a' gets 'a '.
I`m wrong ?
The current behavior looks ok to me. The expression has type CHAR(2)
because the arguments have types CHAR(1) and CHAR(2). So ‘a’ is widened
to ‘a ‘.
Julian
On Sep 22, 2021, at 12:00 AM, stanilovsky evgeny
wrote:
sorry for typo, 'fill the ticket' of course ))
hi community !
I found that COALESCE('a', 'bb') will return 'a ' <-- whitespace is
present, i found that now it expands for maximum presented CHAR
(RelDataType#inferReturnType logic)
sql 92 standard tolds:
'''
COALESCE (V1, V2) is equivalent to the following :
CASE WHEN V1 IS NOT NULL THEN V1 ELSE V2 END
'''
So i suppose that existing behavior is erroneous, if it`s ok i fell
the ticket.
wdyt ?
Re: COALESCE returns maximum of available CHAR capacity.
Julian, thanks !
mysql differs here:
mysql> SELECT LENGTH(a) from (SELECT COALESCE('a', '') as a) as t;
+---+
| LENGTH(a) |
+---+
| 1 |
+---+
mysql> SELECT LENGTH(a) from (SELECT COALESCE('aa', '') as a) as t;
+---+
| LENGTH(a) |
+---+
| 2 |
+---+
I don`t claim that calcite need to be the same, i just want to understand
what is correct.
My point of view - if COALESCE transforms into CASE-WHEN statement and
returns first not null it strange instead of expected 'a' gets 'a '.
I`m wrong ?
The current behavior looks ok to me. The expression has type CHAR(2)
because the arguments have types CHAR(1) and CHAR(2). So ‘a’ is widened
to ‘a ‘.
Julian
On Sep 22, 2021, at 12:00 AM, stanilovsky evgeny
wrote:
sorry for typo, 'fill the ticket' of course ))
hi community !
I found that COALESCE('a', 'bb') will return 'a ' <-- whitespace is
present, i found that now it expands for maximum presented CHAR
(RelDataType#inferReturnType logic)
sql 92 standard tolds:
'''
COALESCE (V1, V2) is equivalent to the following :
CASE WHEN V1 IS NOT NULL THEN V1 ELSE V2 END
'''
So i suppose that existing behavior is erroneous, if it`s ok i fell
the ticket.
wdyt ?
Re: COALESCE returns maximum of available CHAR capacity.
sorry for typo, 'fill the ticket' of course ))
hi community !
I found that COALESCE('a', 'bb') will return 'a ' <-- whitespace is
present, i found that now it expands for maximum presented CHAR
(RelDataType#inferReturnType logic)
sql 92 standard tolds:
'''
COALESCE (V1, V2) is equivalent to the following :
CASE WHEN V1 IS NOT NULL THEN V1 ELSE V2 END
'''
So i suppose that existing behavior is erroneous, if it`s ok i fell the
ticket.
wdyt ?
COALESCE returns maximum of available CHAR capacity.
hi community !
I found that COALESCE('a', 'bb') will return 'a ' <-- whitespace is
present, i found that now it expands for maximum presented CHAR
(RelDataType#inferReturnType logic)
sql 92 standard tolds:
'''
COALESCE (V1, V2) is equivalent to the following :
CASE WHEN V1 IS NOT NULL THEN V1 ELSE V2 END
'''
So i suppose that existing behavior is erroneous, if it`s ok i fell the
ticket.
wdyt ?
Re: Difficulties writing custom parser in SBT build system
hello, you can take as a starting point this PR for example : https://issues.apache.org/jira/browse/IGNITE-13547 Hello Calcite developers, I am trying to add custom types to the Calcite parser. From what I know, it seems that I need configure a config.fmpp file, modify Parser.jj, and run FMPP and JavaCC on them to create a custom parser (with a build system like the one for the Babel parser). However, we use sbt as our build system, whose ecosystem lacks adequate substitutes for the Gradle plugins that the Calcite project uses to generate its parser. Before I go and write these plugins myself, I want to ask the mailing list whether there is an easier way to add custom types when using sbt. Thanks for your time, Tej Qu Nair
Re: Deduplicate correlate variables question.
thanks! done, i hope ) Thanks! Can you improve the JIRA subject, so that everyone reading the release notes will understand it? (I had to look up COLLECTION_TABLE.) On Jun 30, 2021, at 5:12 AM, stanilovsky evgeny wrote: Guys, i fill the ticket [1] and PR is ready for check, can anyone take a look on it ? Additional test appended, all tests locally passed. [1] https://issues.apache.org/jira/browse/CALCITE-4673 thanks ! The PR needs to fix a bug or implement a feature. So, first you should log a JIRA case describing what doesn’t work. Write tests for what doesn’t work that you want to make work. (Or maybe you can refactor/generalize existing tests.) Then submit a PR, and we will review that PR on the basis of whether it fixes the bug. That may sound like a lot of process. But the alternative is people just randomly changing code. No, this sounds like a proper way ) Ok, i understand you ! On Jun 3, 2021, at 7:37 AM, stanilovsky evgeny wrote: Julian, thanks for reply and comments. Can you explain, is it would possible to commit PR containing deduplication code moved upper this flag ? If so - i will create PR and rerun all existing tests, of course. Thanks ! Master is a moving target. Apparently you are using a version of the code from sometime in March. These links work better: [1] https://github.com/apache/calcite/blob/796675c9b33e0461bc45a72780162d474a4b098b/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 [2] https://github.com/apache/calcite/blob/796675c9b33e0461bc45a72780162d474a4b098b/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 I don't know the exact cause of what you are seeing, but when you set "expand=false" you are choosing a newer (and better) code path. We do not have feature parity between the code paths. Nor do we write tests for both code paths. Some day I'd like to officially deprecate, then obsolete, "expand=true". It is de facto deprecated because we put more effort into the "expand=false" path. Julian On Fri, May 28, 2021 at 1:19 AM stanilovsky evgeny wrote: Hi, calciters ) I am trying to figure out why DeduplicateCorrelateVariables [1] is called only if withExpand [2] flag is true ? Why we don`t need to deduplicate in appropriate case ? [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 [2] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 Thanks !
Contributor rights
Hi, Could you please grant me contributor rights in Calcite JIRA? My username is "zstan". Thank you. Evgeniy.
Re: Deduplicate correlate variables question.
Guys, i fill the ticket [1] and PR is ready for check, can anyone take a look on it ? Additional test appended, all tests locally passed. [1] https://issues.apache.org/jira/browse/CALCITE-4673 thanks ! The PR needs to fix a bug or implement a feature. So, first you should log a JIRA case describing what doesn’t work. Write tests for what doesn’t work that you want to make work. (Or maybe you can refactor/generalize existing tests.) Then submit a PR, and we will review that PR on the basis of whether it fixes the bug. That may sound like a lot of process. But the alternative is people just randomly changing code. No, this sounds like a proper way ) Ok, i understand you ! On Jun 3, 2021, at 7:37 AM, stanilovsky evgeny wrote: Julian, thanks for reply and comments. Can you explain, is it would possible to commit PR containing deduplication code moved upper this flag ? If so - i will create PR and rerun all existing tests, of course. Thanks ! Master is a moving target. Apparently you are using a version of the code from sometime in March. These links work better: [1] https://github.com/apache/calcite/blob/796675c9b33e0461bc45a72780162d474a4b098b/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 [2] https://github.com/apache/calcite/blob/796675c9b33e0461bc45a72780162d474a4b098b/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 I don't know the exact cause of what you are seeing, but when you set "expand=false" you are choosing a newer (and better) code path. We do not have feature parity between the code paths. Nor do we write tests for both code paths. Some day I'd like to officially deprecate, then obsolete, "expand=true". It is de facto deprecated because we put more effort into the "expand=false" path. Julian On Fri, May 28, 2021 at 1:19 AM stanilovsky evgeny wrote: Hi, calciters ) I am trying to figure out why DeduplicateCorrelateVariables [1] is called only if withExpand [2] flag is true ? Why we don`t need to deduplicate in appropriate case ? [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 [2] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 Thanks !
[jira] [Created] (CALCITE-4673) Correlate variables deduplication are not triggers under COLLECTION_TABLE.
Stanilovsky Evgeny created CALCITE-4673:
---
Summary: Correlate variables deduplication are not triggers under
COLLECTION_TABLE.
Key: CALCITE-4673
URL: https://issues.apache.org/jira/browse/CALCITE-4673
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.27.0
Reporter: Stanilovsky Evgeny
request like:
{noformat}
select e.deptno, (select * from lateral table(DEDUP(e.deptno, e.deptno))) from
emp e;
{noformat}
shows 2 different correlates, while it only one.
{noformat}
LogicalProject(DEPTNO=[$7], EXPR$1=[$SCALAR_QUERY({
LogicalProject(NAME=[$0])
LogicalTableFunctionScan(invocation=[DEDUP($cor0.DEPTNO, $cor1.DEPTNO)],
rowType=[RecordType(VARCHAR(1024) NAME)])
})])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [QUESTION] Calcite 1.27.0 Potential Sarg problems
Sorry, my fault, seems no problem on calcite side here. Hello ! After 1.27.0 upgrading i found problems with correctness of condition definition. I have simple query : SELECT * FROM products WHERE name = 'Canon' OR category = 'Video' and table like : 5, "Video", 2, "Camera Media", 21, "Media 3" 6, "Video", 2, "Camera Lens", 22, "Lens 3" 7, "Video", 1, null, 0, "Canon" I suppose to obtain plan with 2 scans and UnionAll and its all ok here, but conditions are erroneous: calcite 1.26 cond:=($t1, _UTF-8'Video') cond:AND(=($t5, _UTF-8'Canon'), OR(IS NULL($t1), <>($t1, _UTF-8'Video'))) condition: AND(SEARCH($1, Sarg[_UTF-8'Video':VARCHAR CHARACTER SET "UTF-8"]:VARCHAR CHARACTER SET "UTF-8"), SEARCH($5, Sarg[(-∞.._UTF-8'Canon':VARCHAR CHARACTER SET "UTF-8"), (_UTF-8'Canon':VARCHAR CHARACTER SET "UTF-8"..+∞), null]:VARCHAR CHARACTER SET "UTF-8")) calcite 1.27 cond:AND(=($t5, _UTF-8'Canon'), OR(IS NULL($t1), <>($t1, _UTF-8'Video'))) cond:AND(=($t1, _UTF-8'Video'), OR(IS NULL($t5), <>($t5, _UTF-8'Canon'))) condition: AND(=($1, _UTF-8'Video'), SEARCH($5, Sarg[(-∞.._UTF-8'Canon':VARCHAR CHARACTER SET "UTF-8"), (_UTF-8'Canon':VARCHAR CHARACTER SET "UTF-8"..+∞); NULL AS TRUE]:VARCHAR CHARACTER SET "UTF-8")) thus i obtain incorrect results can someone point me possible issue, code or smt else to start investigation ? thanks !
[QUESTION] Calcite 1.27.0 Potential Sarg problems
Hello ! After 1.27.0 upgrading i found problems with correctness of condition definition. I have simple query : SELECT * FROM products WHERE name = 'Canon' OR category = 'Video' and table like : 5, "Video", 2, "Camera Media", 21, "Media 3" 6, "Video", 2, "Camera Lens", 22, "Lens 3" 7, "Video", 1, null, 0, "Canon" I suppose to obtain plan with 2 scans and UnionAll and its all ok here, but conditions are erroneous: calcite 1.26 cond:=($t1, _UTF-8'Video') cond:AND(=($t5, _UTF-8'Canon'), OR(IS NULL($t1), <>($t1, _UTF-8'Video'))) condition: AND(SEARCH($1, Sarg[_UTF-8'Video':VARCHAR CHARACTER SET "UTF-8"]:VARCHAR CHARACTER SET "UTF-8"), SEARCH($5, Sarg[(-∞.._UTF-8'Canon':VARCHAR CHARACTER SET "UTF-8"), (_UTF-8'Canon':VARCHAR CHARACTER SET "UTF-8"..+∞), null]:VARCHAR CHARACTER SET "UTF-8")) calcite 1.27 cond:AND(=($t5, _UTF-8'Canon'), OR(IS NULL($t1), <>($t1, _UTF-8'Video'))) cond:AND(=($t1, _UTF-8'Video'), OR(IS NULL($t5), <>($t5, _UTF-8'Canon'))) condition: AND(=($1, _UTF-8'Video'), SEARCH($5, Sarg[(-∞.._UTF-8'Canon':VARCHAR CHARACTER SET "UTF-8"), (_UTF-8'Canon':VARCHAR CHARACTER SET "UTF-8"..+∞); NULL AS TRUE]:VARCHAR CHARACTER SET "UTF-8")) thus i obtain incorrect results can someone point me possible issue, code or smt else to start investigation ? thanks !
Re: Deduplicate correlate variables question.
The PR needs to fix a bug or implement a feature. So, first you should log a JIRA case describing what doesn’t work. Write tests for what doesn’t work that you want to make work. (Or maybe you can refactor/generalize existing tests.) Then submit a PR, and we will review that PR on the basis of whether it fixes the bug. That may sound like a lot of process. But the alternative is people just randomly changing code. No, this sounds like a proper way ) Ok, i understand you ! On Jun 3, 2021, at 7:37 AM, stanilovsky evgeny wrote: Julian, thanks for reply and comments. Can you explain, is it would possible to commit PR containing deduplication code moved upper this flag ? If so - i will create PR and rerun all existing tests, of course. Thanks ! Master is a moving target. Apparently you are using a version of the code from sometime in March. These links work better: [1] https://github.com/apache/calcite/blob/796675c9b33e0461bc45a72780162d474a4b098b/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 [2] https://github.com/apache/calcite/blob/796675c9b33e0461bc45a72780162d474a4b098b/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 I don't know the exact cause of what you are seeing, but when you set "expand=false" you are choosing a newer (and better) code path. We do not have feature parity between the code paths. Nor do we write tests for both code paths. Some day I'd like to officially deprecate, then obsolete, "expand=true". It is de facto deprecated because we put more effort into the "expand=false" path. Julian On Fri, May 28, 2021 at 1:19 AM stanilovsky evgeny wrote: Hi, calciters ) I am trying to figure out why DeduplicateCorrelateVariables [1] is called only if withExpand [2] flag is true ? Why we don`t need to deduplicate in appropriate case ? [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 [2] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 Thanks !
Re: Deduplicate correlate variables question.
Julian, thanks for reply and comments. Can you explain, is it would possible to commit PR containing deduplication code moved upper this flag ? If so - i will create PR and rerun all existing tests, of course. Thanks ! Master is a moving target. Apparently you are using a version of the code from sometime in March. These links work better: [1] https://github.com/apache/calcite/blob/796675c9b33e0461bc45a72780162d474a4b098b/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 [2] https://github.com/apache/calcite/blob/796675c9b33e0461bc45a72780162d474a4b098b/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 I don't know the exact cause of what you are seeing, but when you set "expand=false" you are choosing a newer (and better) code path. We do not have feature parity between the code paths. Nor do we write tests for both code paths. Some day I'd like to officially deprecate, then obsolete, "expand=true". It is de facto deprecated because we put more effort into the "expand=false" path. Julian On Fri, May 28, 2021 at 1:19 AM stanilovsky evgeny wrote: Hi, calciters ) I am trying to figure out why DeduplicateCorrelateVariables [1] is called only if withExpand [2] flag is true ? Why we don`t need to deduplicate in appropriate case ? [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 [2] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 Thanks !
Deduplicate correlate variables question.
Hi, calciters ) I am trying to figure out why DeduplicateCorrelateVariables [1] is called only if withExpand [2] flag is true ? Why we don`t need to deduplicate in appropriate case ? [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L2881 [2] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L6209 Thanks !
