[jira] [Commented] (CALCITE-5948) Use explicit casting if element type in ARRAY/MAP does not equal derived component type
[
https://issues.apache.org/jira/browse/CALCITE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786827#comment-17786827
]
Steven Talbot commented on CALCITE-5948:
[~taoran] thanks for the response! Our use case is we parse user-provided SQL
snippets as dimension/measure definitions and use them to build out SQL queries
against various analytical DBs.
We prefer no auto-CASTs like this for two reasons:
# Confusing/unsightly for users when they view the SQL we generate.
# Prefer to not insist on Calcite's strict typing literally showing up in the
SQL, because ensuring every possible type has the right name/semantics in every
supported analytical DB is fraught. DBs do the type unification for you under
the hood if passed a list of e.g. heterogenous numeric types in SQL.
So not a huge deal, but just a nice to have. I can work around this by
transforming the tree that comes out of parse to a subclass of these operators
that removes this behavior, so I'm not blocked.
> Use explicit casting if element type in ARRAY/MAP does not equal derived
> component type
> ---
>
> Key: CALCITE-5948
> URL: https://issues.apache.org/jira/browse/CALCITE-5948
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> First, we need to reach a consensus to allow types of the same family to
> coexist in multiset such as array and map.
> It means the form like `{*}array(1, cast(2 as tinyint)){*}` is correct(the
> LeastRestrictiveType is Integer). However, this function validate success in
> calcite but it failed in runtime, exception stack is:
> {code:java}
> java.lang.ClassCastException: class java.lang.Byte cannot be cast to class
> java.lang.Integer
> at
> org.apache.calcite.avatica.util.AbstractCursor$IntAccessor.getInt(AbstractCursor.java:522)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1396)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1377)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1432)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getString(AbstractCursor.java:1444)
> at
> org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:241)
> at org.apache.calcite.util.JdbcTypeImpl$10.get(JdbcTypeImpl.java:112)
> at org.apache.calcite.util.JdbcTypeImpl$10.get(JdbcTypeImpl.java:109)
> at
> org.apache.calcite.sql.test.ResultCheckers.compareResultSetWithMatcher(ResultCheckers.java:248)
> at
> org.apache.calcite.sql.test.ResultCheckers$MatcherResultChecker.checkResult(ResultCheckers
> {code}
>
> And `{*}map[1, 1, 2, cast(1 as tinyint)]{*}` is correct but calcite throw
> exception:
> {code:java}
> java.lang.AssertionError: Expected query to throw exception, but it did not;
> query [values (map[1, 1, 2, cast(1 as tinyint)])]; expected [Parameters must
> be of the same type]
> at org.apache.calcite.sql.test.SqlTests.checkEx(SqlTests.java:240)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.assertExceptionIsThrown(AbstractSqlTester.java:111)
> at
> org.apache.calcite.test.SqlOperatorFixtureImpl.checkQueryFails(SqlOperatorFixtureImpl.java:174)
> {code}
>
> std ArrayConstructor.
> {code:java}
> public class SqlArrayValueConstructor extends SqlMultisetValueConstructor {
> public SqlArrayValueConstructor() {
> super("ARRAY", SqlKind.ARRAY_VALUE_CONSTRUCTOR);
> }
> @Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> RelDataType type =
> getComponentType(
> opBinding.getTypeFactory(),
> opBinding.collectOperandTypes());
> --> we need explicit cast here
> requireNonNull(type, "inferred array element type");
> return SqlTypeUtil.createArrayType(
> opBinding.getTypeFactory(), type, false);
> }
> } {code}
> std map constructor:
> {code:java}
> public class SqlMapValueConstructor extends SqlMultisetValueConstructor {
> public SqlMapValueConstructor() {
> super("MAP", SqlKind.MAP_VALUE_CONSTRUCTOR);
> }
> @Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> Pair<@Nullable RelDataType, @Nullable RelDataType> type =
> getComponentTypes(
> opBinding.getTypeFactory(), opBinding.collectOperandTypes());
> --> we need explicit cast here
> return SqlTypeUtil.createMapType(
> opBinding.getTypeFactory(),
> requireNonNull(type.left, "inferred key type"),
>
[jira] [Commented] (CALCITE-5948) Use explicit casting if element type in ARRAY/MAP does not equal derived component type
[
https://issues.apache.org/jira/browse/CALCITE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786582#comment-17786582
]
Ran Tao commented on CALCITE-5948:
--
[~swtalbot] hi, thanks for your feedback.
I want to know what your scenario is. As the array function is defined, it
should be a same type. Calcite ensures this by extracting the
biggest/leastRestrictive type of the element on the type side. But the final
array list is a heterogeneous list if no explicit conversion is performed.
> Use explicit casting if element type in ARRAY/MAP does not equal derived
> component type
> ---
>
> Key: CALCITE-5948
> URL: https://issues.apache.org/jira/browse/CALCITE-5948
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> First, we need to reach a consensus to allow types of the same family to
> coexist in multiset such as array and map.
> It means the form like `{*}array(1, cast(2 as tinyint)){*}` is correct(the
> LeastRestrictiveType is Integer). However, this function validate success in
> calcite but it failed in runtime, exception stack is:
> {code:java}
> java.lang.ClassCastException: class java.lang.Byte cannot be cast to class
> java.lang.Integer
> at
> org.apache.calcite.avatica.util.AbstractCursor$IntAccessor.getInt(AbstractCursor.java:522)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1396)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1377)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1432)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getString(AbstractCursor.java:1444)
> at
> org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:241)
> at org.apache.calcite.util.JdbcTypeImpl$10.get(JdbcTypeImpl.java:112)
> at org.apache.calcite.util.JdbcTypeImpl$10.get(JdbcTypeImpl.java:109)
> at
> org.apache.calcite.sql.test.ResultCheckers.compareResultSetWithMatcher(ResultCheckers.java:248)
> at
> org.apache.calcite.sql.test.ResultCheckers$MatcherResultChecker.checkResult(ResultCheckers
> {code}
>
> And `{*}map[1, 1, 2, cast(1 as tinyint)]{*}` is correct but calcite throw
> exception:
> {code:java}
> java.lang.AssertionError: Expected query to throw exception, but it did not;
> query [values (map[1, 1, 2, cast(1 as tinyint)])]; expected [Parameters must
> be of the same type]
> at org.apache.calcite.sql.test.SqlTests.checkEx(SqlTests.java:240)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.assertExceptionIsThrown(AbstractSqlTester.java:111)
> at
> org.apache.calcite.test.SqlOperatorFixtureImpl.checkQueryFails(SqlOperatorFixtureImpl.java:174)
> {code}
>
> std ArrayConstructor.
> {code:java}
> public class SqlArrayValueConstructor extends SqlMultisetValueConstructor {
> public SqlArrayValueConstructor() {
> super("ARRAY", SqlKind.ARRAY_VALUE_CONSTRUCTOR);
> }
> @Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> RelDataType type =
> getComponentType(
> opBinding.getTypeFactory(),
> opBinding.collectOperandTypes());
> --> we need explicit cast here
> requireNonNull(type, "inferred array element type");
> return SqlTypeUtil.createArrayType(
> opBinding.getTypeFactory(), type, false);
> }
> } {code}
> std map constructor:
> {code:java}
> public class SqlMapValueConstructor extends SqlMultisetValueConstructor {
> public SqlMapValueConstructor() {
> super("MAP", SqlKind.MAP_VALUE_CONSTRUCTOR);
> }
> @Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> Pair<@Nullable RelDataType, @Nullable RelDataType> type =
> getComponentTypes(
> opBinding.getTypeFactory(), opBinding.collectOperandTypes());
> --> we need explicit cast here
> return SqlTypeUtil.createMapType(
> opBinding.getTypeFactory(),
> requireNonNull(type.left, "inferred key type"),
> requireNonNull(type.right, "inferred value type"),
> false);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5948) Use explicit casting if element type in ARRAY/MAP does not equal derived component type
[
https://issues.apache.org/jira/browse/CALCITE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786545#comment-17786545
]
Steven Talbot commented on CALCITE-5948:
Looking through the commit, there doesn't seem to be a way to disable this
behavior, right? For our app, we prefer to leave the SQL alone unless
absolutely necessary. There are various hooks in SqlValidator to disable such
rewrites ("withCallRewrite false", or "withTypeCoercionFactory" with
"needToCast" false). But since this fix works by reaching back into the SqlCall
in "inferReturnType" and mutating it outside of the scope of the validator,
none of those configs apply.
I can file an issue if there is indeed no way to disable this. I think this
should be configurable.
> Use explicit casting if element type in ARRAY/MAP does not equal derived
> component type
> ---
>
> Key: CALCITE-5948
> URL: https://issues.apache.org/jira/browse/CALCITE-5948
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> First, we need to reach a consensus to allow types of the same family to
> coexist in multiset such as array and map.
> It means the form like `{*}array(1, cast(2 as tinyint)){*}` is correct(the
> LeastRestrictiveType is Integer). However, this function validate success in
> calcite but it failed in runtime, exception stack is:
> {code:java}
> java.lang.ClassCastException: class java.lang.Byte cannot be cast to class
> java.lang.Integer
> at
> org.apache.calcite.avatica.util.AbstractCursor$IntAccessor.getInt(AbstractCursor.java:522)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1396)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1377)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1432)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getString(AbstractCursor.java:1444)
> at
> org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:241)
> at org.apache.calcite.util.JdbcTypeImpl$10.get(JdbcTypeImpl.java:112)
> at org.apache.calcite.util.JdbcTypeImpl$10.get(JdbcTypeImpl.java:109)
> at
> org.apache.calcite.sql.test.ResultCheckers.compareResultSetWithMatcher(ResultCheckers.java:248)
> at
> org.apache.calcite.sql.test.ResultCheckers$MatcherResultChecker.checkResult(ResultCheckers
> {code}
>
> And `{*}map[1, 1, 2, cast(1 as tinyint)]{*}` is correct but calcite throw
> exception:
> {code:java}
> java.lang.AssertionError: Expected query to throw exception, but it did not;
> query [values (map[1, 1, 2, cast(1 as tinyint)])]; expected [Parameters must
> be of the same type]
> at org.apache.calcite.sql.test.SqlTests.checkEx(SqlTests.java:240)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.assertExceptionIsThrown(AbstractSqlTester.java:111)
> at
> org.apache.calcite.test.SqlOperatorFixtureImpl.checkQueryFails(SqlOperatorFixtureImpl.java:174)
> {code}
>
> std ArrayConstructor.
> {code:java}
> public class SqlArrayValueConstructor extends SqlMultisetValueConstructor {
> public SqlArrayValueConstructor() {
> super("ARRAY", SqlKind.ARRAY_VALUE_CONSTRUCTOR);
> }
> @Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> RelDataType type =
> getComponentType(
> opBinding.getTypeFactory(),
> opBinding.collectOperandTypes());
> --> we need explicit cast here
> requireNonNull(type, "inferred array element type");
> return SqlTypeUtil.createArrayType(
> opBinding.getTypeFactory(), type, false);
> }
> } {code}
> std map constructor:
> {code:java}
> public class SqlMapValueConstructor extends SqlMultisetValueConstructor {
> public SqlMapValueConstructor() {
> super("MAP", SqlKind.MAP_VALUE_CONSTRUCTOR);
> }
> @Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> Pair<@Nullable RelDataType, @Nullable RelDataType> type =
> getComponentTypes(
> opBinding.getTypeFactory(), opBinding.collectOperandTypes());
> --> we need explicit cast here
> return SqlTypeUtil.createMapType(
> opBinding.getTypeFactory(),
> requireNonNull(type.left, "inferred key type"),
> requireNonNull(type.right, "inferred value type"),
> false);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5948) Use explicit casting if element type in ARRAY/MAP does not equal derived component type
[
https://issues.apache.org/jira/browse/CALCITE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773402#comment-17773402
]
Tanner Clary commented on CALCITE-5948:
---
[~taoran] Sounds good, will mark that as resolved from the same commit.
> Use explicit casting if element type in ARRAY/MAP does not equal derived
> component type
> ---
>
> Key: CALCITE-5948
> URL: https://issues.apache.org/jira/browse/CALCITE-5948
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> First, we need to reach a consensus to allow types of the same family to
> coexist in multiset such as array and map.
> It means the form like `{*}array(1, cast(2 as tinyint)){*}` is correct(the
> LeastRestrictiveType is Integer). However, this function validate success in
> calcite but it failed in runtime, exception stack is:
> {code:java}
> java.lang.ClassCastException: class java.lang.Byte cannot be cast to class
> java.lang.Integer
> at
> org.apache.calcite.avatica.util.AbstractCursor$IntAccessor.getInt(AbstractCursor.java:522)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1396)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1377)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1432)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getString(AbstractCursor.java:1444)
> at
> org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:241)
> at org.apache.calcite.util.JdbcTypeImpl$10.get(JdbcTypeImpl.java:112)
> at org.apache.calcite.util.JdbcTypeImpl$10.get(JdbcTypeImpl.java:109)
> at
> org.apache.calcite.sql.test.ResultCheckers.compareResultSetWithMatcher(ResultCheckers.java:248)
> at
> org.apache.calcite.sql.test.ResultCheckers$MatcherResultChecker.checkResult(ResultCheckers
> {code}
>
> And `{*}map[1, 1, 2, cast(1 as tinyint)]{*}` is correct but calcite throw
> exception:
> {code:java}
> java.lang.AssertionError: Expected query to throw exception, but it did not;
> query [values (map[1, 1, 2, cast(1 as tinyint)])]; expected [Parameters must
> be of the same type]
> at org.apache.calcite.sql.test.SqlTests.checkEx(SqlTests.java:240)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.assertExceptionIsThrown(AbstractSqlTester.java:111)
> at
> org.apache.calcite.test.SqlOperatorFixtureImpl.checkQueryFails(SqlOperatorFixtureImpl.java:174)
> {code}
>
> std ArrayConstructor.
> {code:java}
> public class SqlArrayValueConstructor extends SqlMultisetValueConstructor {
> public SqlArrayValueConstructor() {
> super("ARRAY", SqlKind.ARRAY_VALUE_CONSTRUCTOR);
> }
> @Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> RelDataType type =
> getComponentType(
> opBinding.getTypeFactory(),
> opBinding.collectOperandTypes());
> --> we need explicit cast here
> requireNonNull(type, "inferred array element type");
> return SqlTypeUtil.createArrayType(
> opBinding.getTypeFactory(), type, false);
> }
> } {code}
> std map constructor:
> {code:java}
> public class SqlMapValueConstructor extends SqlMultisetValueConstructor {
> public SqlMapValueConstructor() {
> super("MAP", SqlKind.MAP_VALUE_CONSTRUCTOR);
> }
> @Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> Pair<@Nullable RelDataType, @Nullable RelDataType> type =
> getComponentTypes(
> opBinding.getTypeFactory(), opBinding.collectOperandTypes());
> --> we need explicit cast here
> return SqlTypeUtil.createMapType(
> opBinding.getTypeFactory(),
> requireNonNull(type.left, "inferred key type"),
> requireNonNull(type.right, "inferred value type"),
> false);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5948) Use explicit casting if element type in ARRAY/MAP does not equal derived component type
[
https://issues.apache.org/jira/browse/CALCITE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773401#comment-17773401
]
Ran Tao commented on CALCITE-5948:
--
[~tanclary] thanks Tanner for patient reviewing!
btw, I think we can safely close CALCITE-4850 either. Because that issue
belongs to a specific case of this ticket.
> Use explicit casting if element type in ARRAY/MAP does not equal derived
> component type
> ---
>
> Key: CALCITE-5948
> URL: https://issues.apache.org/jira/browse/CALCITE-5948
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.36.0
>
>
> First, we need to reach a consensus to allow types of the same family to
> coexist in multiset such as array and map.
> It means the form like `{*}array(1, cast(2 as tinyint)){*}` is correct(the
> LeastRestrictiveType is Integer). However, this function validate success in
> calcite but it failed in runtime, exception stack is:
> {code:java}
> java.lang.ClassCastException: class java.lang.Byte cannot be cast to class
> java.lang.Integer
> at
> org.apache.calcite.avatica.util.AbstractCursor$IntAccessor.getInt(AbstractCursor.java:522)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1396)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1377)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1432)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getString(AbstractCursor.java:1444)
> at
> org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:241)
> at org.apache.calcite.util.JdbcTypeImpl$10.get(JdbcTypeImpl.java:112)
> at org.apache.calcite.util.JdbcTypeImpl$10.get(JdbcTypeImpl.java:109)
> at
> org.apache.calcite.sql.test.ResultCheckers.compareResultSetWithMatcher(ResultCheckers.java:248)
> at
> org.apache.calcite.sql.test.ResultCheckers$MatcherResultChecker.checkResult(ResultCheckers
> {code}
>
> And `{*}map[1, 1, 2, cast(1 as tinyint)]{*}` is correct but calcite throw
> exception:
> {code:java}
> java.lang.AssertionError: Expected query to throw exception, but it did not;
> query [values (map[1, 1, 2, cast(1 as tinyint)])]; expected [Parameters must
> be of the same type]
> at org.apache.calcite.sql.test.SqlTests.checkEx(SqlTests.java:240)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.assertExceptionIsThrown(AbstractSqlTester.java:111)
> at
> org.apache.calcite.test.SqlOperatorFixtureImpl.checkQueryFails(SqlOperatorFixtureImpl.java:174)
> {code}
>
> std ArrayConstructor.
> {code:java}
> public class SqlArrayValueConstructor extends SqlMultisetValueConstructor {
> public SqlArrayValueConstructor() {
> super("ARRAY", SqlKind.ARRAY_VALUE_CONSTRUCTOR);
> }
> @Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> RelDataType type =
> getComponentType(
> opBinding.getTypeFactory(),
> opBinding.collectOperandTypes());
> --> we need explicit cast here
> requireNonNull(type, "inferred array element type");
> return SqlTypeUtil.createArrayType(
> opBinding.getTypeFactory(), type, false);
> }
> } {code}
> std map constructor:
> {code:java}
> public class SqlMapValueConstructor extends SqlMultisetValueConstructor {
> public SqlMapValueConstructor() {
> super("MAP", SqlKind.MAP_VALUE_CONSTRUCTOR);
> }
> @Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
> Pair<@Nullable RelDataType, @Nullable RelDataType> type =
> getComponentTypes(
> opBinding.getTypeFactory(), opBinding.collectOperandTypes());
> --> we need explicit cast here
> return SqlTypeUtil.createMapType(
> opBinding.getTypeFactory(),
> requireNonNull(type.left, "inferred key type"),
> requireNonNull(type.right, "inferred value type"),
> false);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
