[
https://issues.apache.org/jira/browse/CALCITE-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720002#comment-17720002
]
Abhishek Singh Chouhan commented on CALCITE-5690:
-------------------------------------------------
[~julianhyde] Any thoughts on what the right thing to do here is? We're caching
DynamicRecordTypeImpl which is mutable. We could look into not caching this or
making the fields list object in RelDataTypeHolder something like a
CopyOnWriteArraylist
> ConcurrentModificationException during validation of table with
> DynamicRecordType
> ---------------------------------------------------------------------------------
>
> Key: CALCITE-5690
> URL: https://issues.apache.org/jira/browse/CALCITE-5690
> Project: Calcite
> Issue Type: Bug
> Affects Versions: 1.32.0
> Reporter: Abhishek Singh Chouhan
> Priority: Major
>
> When multiple threads are doing validation on a Table with
> DynamicRecordTypeImpl, we run into a ConcurrentModificationException. One of
> the instances where this happens is when two validations are happening in
> parallel
> Thread1 -
> SELECT DS_GET_QUANTILE(DS_QUANTILES_SKETCH(COL1), 0.9) FROM TABLE1 GROUP BY
> COL1, COL2, COL3
> Thread2 - SELECT DS_GET_QUANTILE(DS_QUANTILES_SKETCH(COL1), 0.9) FROM TABLE1
> GROUP BY COL1, COL2
> While expanding the groupby in thread1 we intern the type at multiple places,
> eg. DelegatingScope#fullyQualify -> SqlValidatorScope#rowType ->
> RelDataTypeFactoryImpl#createTypeWithNullability -> canonize
> During the validation in thread1 we cache DynamicRecordTypeImpl with Col1 and
> Col2 as the fields.
> The race condition happens when thread2 gets a hold of this(abovementioned)
> cached DynamicRecordTypeImpl during its validation and is iterating over the
> fields eg during SqlValidatorImpl.validateGroupItem and thread 1 then goes
> and inserts Col3 field in the holder.
> {code:java}
> java.util.ConcurrentModificationException
> at
> java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
> at
> org.apache.calcite.rel.type.RelDataTypeHolder.getFieldOrInsert(RelDataTypeHolder.java:56)
> at
> org.apache.calcite.rel.type.DynamicRecordTypeImpl.getField(DynamicRecordTypeImpl.java:59)
> at
> org.apache.calcite.sql.validate.SqlNameMatchers$BaseMatcher.field(SqlNameMatchers.java:126)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6455)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6360)
> at org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:324)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1869)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1854)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateGroupItem(SqlValidatorImpl.java:4393)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateGroupClause(SqlValidatorImpl.java:4366)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3701)
> at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)
> at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1107)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1078)
> at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:248)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1053)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:759)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
