Jenkins build is back to normal : Calcite » Calcite-snapshots #900

2024-06-04 Thread Apache Jenkins Server 
See

Build failed in Jenkins: Calcite » Calcite-snapshots #899

2024-06-04 Thread Apache Jenkins Server 
See 


Changes:

[mbudiu] [CALCITE-6422] Query with '<>' throws NullPointerException during 
materialized view matching


--
[...truncated 473.26 KB...]
SqlToRelConverterExtendedTest > testGroupJustOneAgg() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlToRelConverterExtendedTest > testCorrelationJoinRex() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

CoreSqlOperatorTest > testMinusDateOperator() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlOperatorUnparseTest > testCastToExactNumeric(CastType, SqlOperatorFixture) > 
[3] TRY_CAST, 
org.apache.calcite.test.SqlOperatorFixtures$SqlOperatorFixtureInvocationHandler@68bff4a5
 STANDARD_OUT
Warning: JDBC driver threw non-SQLException

CoreSqlOperatorTest > testJsonInsert() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

CoreSqlOperatorTest > testArrayJoinFunc() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

CoreSqlOperatorTest STANDARD_OUT
Warning: JDBC driver threw non-SQLException

CoreSqlOperatorTest > testExistsFunc() STANDARD_OUT
Warning: JDBC driver threw non-SQLException
  7.5sec, org.apache.calcite.test.JdbcTest > 
testOrderByVarious()

JdbcTest STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlToRelConverterExtendedTest > testSelectWithoutFrom() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlToRelConverterExtendedTest > testWithInsideScalarSubQueryRex() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlToRelConverterExtendedTest STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlToRelConverterExtendedTest > testGroupingSetsWithCube() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlToRelConverterExtendedTest > testTableFunctionWithParamNames() STANDARD_OUT
Warning: JDBC driver threw non-SQLException
  4.0sec,3 completed,   0 failed,   0 skipped, 
org.apache.calcite.test.SqlOperatorUnparseTest > 
testCastToExactNumeric(CastType, SqlOperatorFixture)

SqlOperatorUnparseTest > testTrimFunc() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlToRelConverterExtendedTest STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlToRelConverterExtendedTest > testJsonValueExpressionOperator() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlToRelConverterExtendedTest > testSelectDistinctGroup() STANDARD_OUT
Warning: JDBC driver threw non-SQLException

SqlToRelConverterExtendedTest > testGroupByAliasOfSubExpressionsInProject() 
STANDARD_OUT
Warning: JDBC driver threw non-SQLException
 16.8sec, org.apache.calcite.test.CoreQuidemTest > 
test(String)[4], [4] sql/cast-with-format.iq
WARNING  22.9sec,  589 completed,   0 failed,   2 
skipped, org.apache.calcite.test.SqlToRelConverterExtendedTest
  6.8sec,3 completed,   0 failed,   0 skipped, 
org.apache.calcite.test.CalciteSqlOperatorTest > 
testCastToExactNumeric(CastType, SqlOperatorFixture)
 10.4sec, org.apache.calcite.test.JdbcTest > 
testAggMultipleMeasures()
WARNING  41.0sec,  359 completed,   0 failed,  18 
skipped, org.apache.calcite.test.JdbcTest
  0.6sec,2 completed,   0 failed,   0 skipped, 
org.apache.calcite.test.CoreSqlOperatorTest > 
testTimestampDiff(boolean)
  0.1sec,3 completed,   0 failed,   0 skipped, 
org.apache.calcite.test.CoreSqlOperatorTest > 
testCastToInterval(CastType, SqlOperatorFixture)
  0.7sec,3 completed,   0 failed,   0 skipped, 
org.apache.calcite.test.CoreSqlOperatorTest > 
testCastExactNumericLimits(CastType, SqlOperatorFixture)
  0.2sec,3 completed,   0 failed,   0 skipped, 
org.apache.calcite.test.CoreSqlOperatorTest > 
testCastDateTime(CastType, SqlOperatorFixture)
  0.0sec,3 completed,   0 failed,   0 skipped, 
org.apache.calcite.test.CoreSqlOperatorTest > 
testCastFormatClauseStringToDateTime(CastType, SqlOperatorFixture)
  0.1sec,3 completed,   0 failed,   0 skipped, 
org.apache.calcite.test.CoreSqlOperatorTest > 
testCastTruncates(CastType, SqlOperatorFixture)
  0.1sec,3 completed,   0 failed,   0 skipped, 
org.apache.calcite.test.CoreSqlOperatorTest > 
testCastDecimalToDoubleToInteger(CastType, SqlOperatorFixture)
  4.9sec, org.apache.calcite.test.CoreQuidemTest > 
test(String)[5], [5] sql/within-distinct.iq
  0.8sec,3 completed,   0 failed,   0 skipped,

Re: Casting Decimal literals problem

2024-06-04 Thread Mihai Budiu 
This is a very old issue in Calcite.
I hope to be able to merge the fix soon: 
https://github.com/apache/calcite/pull/3733
However, the fix itself has uncovered a bunch of other issues, so things are 
not quite so straightforward.

Mihai

From: Alessandro Solimando 
Sent: Tuesday, June 4, 2024 4:40 AM
To: n...@sqreamtech.com.invalid 
Cc: dev@calcite.apache.org 
Subject: Re: Casting Decimal literals problem

Hi Neo,
this is a known issue, you can check CALCITE-6322
 and related tickets.

Best regards,
Alessandro

On Tue, 4 Jun 2024 at 11:58, Neo Michael 
wrote:

> Hello Calcite devs,
> I have come across some strange behavior when Decimal literals are being
> casted to a different precision and scale.  For example for something like:
>
> : jdbc:calcite:model=src/test/resources/mode> select cast(1.11 as
> numeric(3,1)); ++ | EXPR$0 | ++ | 1.11 | ++ 1 row
> selected (0.509 seconds)
>
> Is this not the wrong result? No casting takes place, the actual cast is
> removed, the literal is given the new RelDataType of Numeric(3, 1), but the
> actual Java BigDecimal is not modified (the java object has precision 3 and
> scale 2). Unless I am misunderstanding the semantics of Numeric, 1.11 is
> not representable in type Numeric(3,1). This is using Calcite
> 1.38.0-SNAPSHOT
>
> I have seen more issues when repeated casts are removed (deemed
> superfluous by the simplifier), but it may be the same issue so let's deal
> with this simpler case first.
>
> Thank you for any help,
> Neo
>

Re: arrow-adapter component creation in Jira

2024-06-04 Thread Alessandro Solimando 
Thanks Michael, understood!

Best regards,
Alessandro

On Tue, Jun 4, 2024, 18:40 Michael Mior  wrote:

> Alessandro,
>
> I believe only PMC members can do this. I created the arrow-adapter
> component.
> --
> Michael Mior
> mm...@apache.org
>
>
> On Tue, Jun 4, 2024 at 12:05 PM Alessandro Solimando <
> alessandro.solima...@gmail.com> wrote:
>
> > Hello,
> > it seems I don't have right to create new "components" in Jira.
> >
> > The arrow adapter has been contributed a while back, it would be nice to
> > have it available as a component in Jira.
> >
> > How does that work in general? Who has the right to create it?
> >
> > Best regards,
> > Alessandro
> >
>

Re: arrow-adapter component creation in Jira

2024-06-04 Thread Michael Mior 
Alessandro,

I believe only PMC members can do this. I created the arrow-adapter
component.
--
Michael Mior
mm...@apache.org


On Tue, Jun 4, 2024 at 12:05 PM Alessandro Solimando <
alessandro.solima...@gmail.com> wrote:

> Hello,
> it seems I don't have right to create new "components" in Jira.
>
> The arrow adapter has been contributed a while back, it would be nice to
> have it available as a component in Jira.
>
> How does that work in general? Who has the right to create it?
>
> Best regards,
> Alessandro
>

arrow-adapter component creation in Jira

2024-06-04 Thread Alessandro Solimando 
Hello,
it seems I don't have right to create new "components" in Jira.

The arrow adapter has been contributed a while back, it would be nice to
have it available as a component in Jira.

How does that work in general? Who has the right to create it?

Best regards,
Alessandro

[jira] [Created] (CALCITE-6429) Arrow adapter should default to the Enumerable convention for unsupported features

2024-06-04 Thread Alessandro Solimando (Jira) 
Alessandro Solimando created CALCITE-6429:
-

 Summary: Arrow adapter should default to the Enumerable convention 
for unsupported features
 Key: CALCITE-6429
 URL: https://issues.apache.org/jira/browse/CALCITE-6429
 Project: Calcite
  Issue Type: Sub-task
Affects Versions: 1.37.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
 Fix For: 1.38.0


Currently, the adapter fails with different errors when one of the known 
unsupported features is used, while it might simply catch the 
"UnsupportedOperationException" and bail out, thus allowing to generate a plan 
where the unsupported operation is implemented in the Enumerable convention.

 

For instance, currently several tests are disabled for this reason, like 
[testArrowProjectFieldsWithDisjunctiveFilter|https://github.com/apache/calcite/blob/be9b860ebf3143dcbdbd0ee1777e604f0ace7b3c/arrow/src/test/java/org/apache/calcite/adapter/arrow/ArrowAdapterTest.java#L242],
 because it would fail as follows:
{code:java}
Error while executing SQL "select "intField", "stringField"
from arrowdata
where "intField"=12 or "stringField"='12'": Error while applying rule 
ArrowFilterRule, args 
[rel#31:LogicalFilter.NONE.[](input=RelSubset#15,condition=OR(=($0, 12), =($1, 
'12'))), rel#1:ArrowTableScan.ARROW.[](table=[ARROW, ARROWDATA],fields=[0, 1, 
2, 3])]
java.sql.SQLException: Error while executing SQL "select "intField", 
"stringField"
from arrowdata
where "intField"=12 or "stringField"='12'": Error while applying rule 
ArrowFilterRule, args 
[rel#31:LogicalFilter.NONE.[](input=RelSubset#15,condition=OR(=($0, 12), =($1, 
'12'))), rel#1:ArrowTableScan.ARROW.[](table=[ARROW, ARROWDATA],fields=[0, 1, 
2, 3])]
    at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
    at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
    at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
    at 
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:228)
    at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:566)
    at 
org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1495)
    at 
org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1434)
    at 
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1493)
    at 
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1483)
    at 
org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1446)
    at 
org.apache.calcite.adapter.arrow.ArrowAdapterTest.testArrowProjectFieldsWithDisjunctiveFilter(ArrowAdapterTest.java:260)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at 
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
    at 
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
    at 
org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
    at 
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
    at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
    at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
    at

Re: Casting Decimal literals problem

2024-06-04 Thread Alessandro Solimando 
Hi Neo,
this is a known issue, you can check CALCITE-6322
 and related tickets.

Best regards,
Alessandro

On Tue, 4 Jun 2024 at 11:58, Neo Michael 
wrote:

> Hello Calcite devs,
> I have come across some strange behavior when Decimal literals are being
> casted to a different precision and scale.  For example for something like:
>
> : jdbc:calcite:model=src/test/resources/mode> select cast(1.11 as
> numeric(3,1)); ++ | EXPR$0 | ++ | 1.11 | ++ 1 row
> selected (0.509 seconds)
>
> Is this not the wrong result? No casting takes place, the actual cast is
> removed, the literal is given the new RelDataType of Numeric(3, 1), but the
> actual Java BigDecimal is not modified (the java object has precision 3 and
> scale 2). Unless I am misunderstanding the semantics of Numeric, 1.11 is
> not representable in type Numeric(3,1). This is using Calcite
> 1.38.0-SNAPSHOT
>
> I have seen more issues when repeated casts are removed (deemed
> superfluous by the simplifier), but it may be the same issue so let's deal
> with this simpler case first.
>
> Thank you for any help,
> Neo
>

Casting Decimal literals problem

2024-06-04 Thread Neo Michael 
Hello Calcite devs,
I have come across some strange behavior when Decimal literals are being
casted to a different precision and scale.  For example for something like:

: jdbc:calcite:model=src/test/resources/mode> select cast(1.11 as
numeric(3,1)); ++ | EXPR$0 | ++ | 1.11 | ++ 1 row
selected (0.509 seconds)

Is this not the wrong result? No casting takes place, the actual cast is
removed, the literal is given the new RelDataType of Numeric(3, 1), but the
actual Java BigDecimal is not modified (the java object has precision 3 and
scale 2). Unless I am misunderstanding the semantics of Numeric, 1.11 is
not representable in type Numeric(3,1). This is using Calcite
1.38.0-SNAPSHOT

I have seen more issues when repeated casts are removed (deemed
superfluous by the simplifier), but it may be the same issue so let's deal
with this simpler case first.

Thank you for any help,
Neo

Re: EnumerableTableScan array/multiset handling

2024-06-04 Thread Christian Beikov 

Hi Julian,

it seems to me that 
org.apache.calcite.rel.type.RelDataTypeFactoryImpl.JavaType, which is a 
subtype of RelDataType, would be the best place to model this. How about 
I add an Accessor contract and a field to JavaType to take care of 
producing expressions for the enumerable adapater? By default, this will 
use an implementation based on public fields or record accessors 
depending on class type. In addition, I would also add an implementation 
that works based on calling getter methods, since that is what I need 
for my purposes.


Wdyt?

Regards,

Christian

Am 31.05.2024 um 03:02 schrieb Julian Hyde:

Thanks for doing these experiments, Christian, and documenting what you found.

I think you’re running into the limitations of ReflectiveSchema. It works with 
POJOs (java classes with public fields) but hasn’t been tested for richer 
variations and therefore just doesn’t work. In many cases, it can be fixed 
(seehttps://issues.apache.org/jira/browse/CALCITE-6244  for example).

I’m uneasy about extending RelDataType to return JavaRowFormat. That seems to 
be introducing a physical property into a logical data type; also, it is 
surfacing the details of one particular adapter (enumerable). Maybe there is a 
way to surface this information via annotations or the java.sql.Wrapper 
interface without extending RelDataType.

Julian



On May 24, 2024, at 11:33 AM, Christian Beikov  
wrote:

Hello,

in my recent experiments I ran into some issues when trying to unnest an array 
of struct.

The query is roughly this: select t.id, e.value1 from MyTable t, 
unnest(t.structArray) e

EnumerableTableScan#fieldExpression will then try to generate code that converts the value of the 
"structArray" column to a List, which is where the problems start to arise. This 
code does not seem to be tested at all, because currently generates a compile error, due to missing a 
cast to "Iterable". It also assumes the data is already available in the JavaRowFormat.CUSTOM 
representation, but AFAIU, it could be in any format.

When using RelRecordType for structs, regular struct columns seem to expect 
JavaRowFormat.ARRAY, but struct arrays don't seem to behave the same way.

What is the expected data format that an enumerator should return for struct 
arrays that are typed as RelRecordType?

To support formats per type it might be nice to allow specifying the 
JavaRowFormat on RelDataType. Wdyt?

Also, is there a way to allow working with custom Java types for table/struct 
rows? From looking at AbstractCursor#createAccessor, it seems the Aviatica code 
currently only works with classes that expose public fields.

Regards,

Christian