[jira] [Created] (CALCITE-6211) SUBSTRING with Integer.MIN_VALUE as a second parameter raise unexpected exception

[Evgeny Stanilovsky (Jira)]

Evgeny Stanilovsky created CALCITE-6211:
---

 Summary: SUBSTRING with Integer.MIN_VALUE as a second parameter 
raise unexpected exception
 Key: CALCITE-6211
 URL: https://issues.apache.org/jira/browse/CALCITE-6211
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.36.0
Reporter: Evgeny Stanilovsky


SUBSTRING function with minimal possible integer offset parameter raise 
exception.

Simple way to reproduce :
append into *SqlOperatorTest* smth like:

{noformat}
f.checkScalar(
String.format("{fn SUBSTRING('abcdef', %d)}", Integer.MIN_VALUE),
"abcdef",
"VARCHAR(6) NOT NULL");

{noformat}

And exception will raised:

{noformat}
String index out of range: -2147483641
{noformat}

it`s all due to integer overflow, check implementation near:

{noformat}
  public static String substring(String c, int s) {
final int s0 = s - 1;  // -2147483648 - 1 = 2147483647
if (s0 <= 0) { // this check is broken
  return c;
}
if (s > c.length()) {
  return "";
}
return c.substring(s0);
  }
{noformat}






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: javadoc in chinese

[Benchao Li]

Thanks Guillaume for reporting this,

Calcite hosts the javadoc here[1], it's in English.

For the docs hosted in javadoc.io, it's in Chinese indeed. It seems to
be a problem of the locale setting while preparing the artifacts for
releasing, and this is an ignored process in our releasing steps in
the past (making sure the locale is en). I'll open a PR to add this
step in our documentation.

[1] https://calcite.apache.org/javadocAggregate/

Guillaume Masse  于2024年1月18日周四 04:36写道：
>
> It looks like the javadoc for 1.36.0 was publish in chinese:
> https://javadoc.io/doc/org.apache.calcite/calcite-core/latest/index.html
>
> --
> Guillaume Massé
> [Gee-OHM]
> (马赛卫)



-- 

Best,
Benchao Li

javadoc in chinese

[Guillaume Masse]

It looks like the javadoc for 1.36.0 was publish in chinese:
https://javadoc.io/doc/org.apache.calcite/calcite-core/latest/index.html

-- 
Guillaume Massé
[Gee-OHM]
(马赛卫)

[jira] [Created] (CALCITE-6210) Cast to VARBINARY causes an assertion failure

[Mihai Budiu (Jira)]

Mihai Budiu created CALCITE-6210:


 Summary: Cast to VARBINARY causes an assertion failure
 Key: CALCITE-6210
 URL: https://issues.apache.org/jira/browse/CALCITE-6210
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.36.0
Reporter: Mihai Budiu


This test in SqlOperatorTest:
{code:java}
 SqlOperatorFixture f = fixture();
 f.checkScalar("CAST('00' AS VARBINARY)", "00", "VARBINARY NOT NULL");
{code}

Causes the following assertion failure:

{code}
java.lang.AssertionError: value 00 does not match type class  
org.apache.calcite.avatica.util.ByteString
at 
org.apache.calcite.linq4j.tree.ConstantExpression.(ConstantExpression.java:51)
at 
org.apache.calcite.linq4j.tree.Expressions.constant(Expressions.java:585)
at 
org.apache.calcite.linq4j.tree.OptimizeShuttle.visit(OptimizeShuttle.java:305)
at 
org.apache.calcite.linq4j.tree.UnaryExpression.accept(UnaryExpression.java:39)
at 
org.apache.calcite.linq4j.tree.TernaryExpression.accept(TernaryExpression.java:47)
at 
org.apache.calcite.linq4j.tree.DeclarationStatement.accept(DeclarationStatement.java:45)
at 
org.apache.calcite.linq4j.tree.DeclarationStatement.accept(DeclarationStatement.java:27)
at 
org.apache.calcite.linq4j.tree.BlockBuilder.optimize(BlockBuilder.java:426)
at 
org.apache.calcite.linq4j.tree.BlockBuilder.toBlock(BlockBuilder.java:340)
at 
org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:102)
at 
org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:68)
at 
org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:133)
at 
org.apache.calcite.rex.RexSimplify.simplifyCast(RexSimplify.java:2272)
at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:292)
at 
org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:250)
at 
org.apache.calcite.rex.RexSimplify.simplifyPreservingType(RexSimplify.java:189)
at 
org.apache.calcite.rex.RexSimplify.simplifyPreservingType(RexSimplify.java:184)
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] Does web site use Google analytics?

[Michael Mior]

Worth noting that using Google Analytics is possible in compliance with the
GDPR if configured appropriately. That said, although we could fix any
configuration issues or find an alternative, I think it would also be fine
to simply remove any analytics from the site altogether.

--
Michael Mior
mm...@apache.org


On Wed, Jan 17, 2024 at 1:05 PM Julian Hyde  wrote:

> Does Calcite’s web site use Google analytics?
>
> If so, we should fix it. Google analytics collects information that is not
> allowed by GDPR. Infra is considering blocking Google analytics for all *.
> apache.org  sites, and if they do that it may make
> our site unusable.
>
> Whimsy has a check on all project sites [1] and it finds one ‘external
> resource’:
>
> > Found 1 external resources: {"fonts.googleapis.com"=>1}
>
> Julian
>
> [1] https://whimsy.apache.org/site/
>
>

Re: [DISCUSS] Does web site use Google analytics?

[Sergey Nuyanzin]

Seems like that
I found this usage [1] of it

What are the alternatives that could be used there?

[1] https://github.com/apache/calcite/blob/main/site/_includes/top.html#L8

On Wed, Jan 17, 2024 at 7:05 PM Julian Hyde  wrote:

> Does Calcite’s web site use Google analytics?
>
> If so, we should fix it. Google analytics collects information that is not
> allowed by GDPR. Infra is considering blocking Google analytics for all *.
> apache.org  sites, and if they do that it may make
> our site unusable.
>
> Whimsy has a check on all project sites [1] and it finds one ‘external
> resource’:
>
> > Found 1 external resources: {"fonts.googleapis.com"=>1}
>
> Julian
>
> [1] https://whimsy.apache.org/site/
>
>

-- 
Best regards,
Sergey

[DISCUSS] Does web site use Google analytics?

[Julian Hyde]

Does Calcite’s web site use Google analytics?

If so, we should fix it. Google analytics collects information that is not 
allowed by GDPR. Infra is considering blocking Google analytics for all 
*.apache.org  sites, and if they do that it may make our 
site unusable.

Whimsy has a check on all project sites [1] and it finds one ‘external 
resource’:

> Found 1 external resources: {"fonts.googleapis.com"=>1}

Julian

[1] https://whimsy.apache.org/site/ 

[jira] [Created] (CALCITE-6209) All queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes

[Konstantin (Jira)]

Konstantin created CALCITE-6209:
---

 Summary: All queries are failing with 
"java.net.SocketTimeoutException: Read timed out" after 3 minutes
 Key: CALCITE-6209
 URL: https://issues.apache.org/jira/browse/CALCITE-6209
 Project: Calcite
  Issue Type: Bug
  Components: avatica
Affects Versions: avatica-1.24.0
Reporter: Konstantin


All queries are failing with "java.net.SocketTimeoutException: Read timed out" 
after 3 minutes.
I tried to change it via +http_connection_timeout+ property but it didn't help.
Looks like that property is related to connect timeout only, but not to request 
timeout.
I checked source code but didn't found any ways to change it there.
Looks like it's always using default timeout=3minutes.
Stack-trace:
{code:java}
org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing 
SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed 
out
at 
org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
at 
org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
at 
org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
at 
org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
at 
org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
at 
org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
at 
org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
at 
org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
at 
org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
at 
org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
at 
org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1) 
FROM trace": java.net.SocketTimeoutException: Read timed out
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
at 
org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
at 
org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
at 
org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
... 12 more
Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read 
timed out
at 
org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
at 
org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
at 
org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
at 
org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
at 
org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
at 
org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
at 
org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
at 
org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
at 
org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
at 
org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
at 
org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:98)
at 
org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:186)
at 
org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:669)
at 
org.apache.calcite.avatica.remote.RemoteMeta$13.call(RemoteMeta.java:283)
at 
org.apache.calcite.avatica.remote.RemoteMeta$13.call(RemoteMeta.java:263)
at 
org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
at 
org.apache.calcite.avatica.remote.RemoteMeta.prepareAndExecute(RemoteMeta.java:262)
at 
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)
at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:157)

Implementing a SqlDialect for PostGIS

[Bertil Chapuis]

Hello Everyone,

Calcite implements support for spatial types (geometry, point, etc.) and 
spatial functions (ST_), and it can connect to PostGIS via a JdbcSchema. 
However, the Postgresql dialect does not currently handle spatial types and 
functions. As a result, Calcite tries to execute the spatial functions at the 
level of the JVM instead of pushing them down to postgis.

As a result, the following query gets executed, but the type of the geom column 
is incorrect:
SELECT id, geom FROM public.spatial_table

The following query fails with a ClassCastException as Calcite tries to use the 
java implementation of the ST_SRID function:
SELECT id, ST_SRID(geom) FROM public.spatial_table
java.lang.ClassCastException: class org.postgresql.util.PGobject cannot be cast 
to class org.locationtech.jts.geom.Geometry (org.postgresql.util.PGobject and 
org.locationtech.jts.geom.Geometry are in unnamed module of loader 'app')

In my current understanding, this issue could be addressed with a new 
PostgisSqlDialect that extends PostgresqlSqlDialect and adds support for 
spatial types and functions. Here is a tentative roadmap:
- Add all the spatial functions to the SqlKind class
- Create a PostgisSqlDialect class that extends PostgresqlSqlDialect
- Add support for the spatial types (geometry) by overriding the getCastSpec 
method of the SqlDialect class
- Add support for the spatial functions by overriding the supportsFunction 
method of the SqlDialect class
- Add support for the spatial aggregate functions by overriding the 
supportsAggregateFunction method of the SqlDialect class

Could someone confirm that overall this approach makes sense? If so, I will 
create a JIRA issue and submit pull requests.

Something that I am not sure about, is how to test the dialect. In my 
understanding, when it comes to niche dialects such as PostGIS, Calcite relies 
on its users to report issues. Is that correct?

Best,

Bertil


signature.asc
Description: Message signed with OpenPGP

[jira] [Created] (CALCITE-6208) JSON_VALUE 'RETURNING' syntax support for arrays with nullable elements

[Clint Wylie (Jira)]

Clint Wylie created CALCITE-6208:


 Summary: JSON_VALUE 'RETURNING' syntax support for arrays with 
nullable elements
 Key: CALCITE-6208
 URL: https://issues.apache.org/jira/browse/CALCITE-6208
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Clint Wylie


The JSON_VALUE function RETURNING syntax is not currently super array friendly 
because it creates an array type with non-null elements. This is primarily due 
to the interaction between SqlDataTypeSpec which creates a not null array with 
not null elements, and the 
SqlJsonValueFunction which forces nullable of the type, but that in turn does 
not force nullable for the element type, so it results in a nullable array but 
still with not null elements.
 
In a lot of cases this isn't a huge problem, but coupled with unnest and 
filtering, it results in filters being incorrectly eliminated after being 
incorrectly classified as 'all true' or 'all false'.
For example, given a query such as

 
{code:java}
select c
from table, unnest(json_value(nested, '$.c' returning bigint array)) as u(c)
where c is not null{code}
c is inferred to be 'BIGINT NOT NULL', which results in 'c is not null' being 
dropped.

I believe the solution to this is to modify the return type inference to 
include special handling for when the SqlDataTypeSpec is an array type to force 
both the array and its elements to be nullable. 

I tested this out and it appears to work as expected.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)