[ https://issues.apache.org/jira/browse/BEAM-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Pilloud updated BEAM-9666: --------------------------------- Status: Open (was: Triage Needed) > UNION ALL with double IllegalStateException > ------------------------------------------- > > Key: BEAM-9666 > URL: https://issues.apache.org/jira/browse/BEAM-9666 > Project: Beam > Issue Type: Bug > Components: dsl-sql-zetasql > Reporter: Andrew Pilloud > Priority: Major > Labels: zetasql-compliance > > Two failures in shard 18 > {code:java} > java.lang.IllegalStateException: the keyCoder of a GroupByKey must be > deterministic > at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:234) > at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:110) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493) > at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:368) > at > org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:118) > at > org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:71) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:476) > at > org.apache.beam.sdk.transforms.join.KeyedPCollectionTuple.apply(KeyedPCollectionTuple.java:108) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSetOperatorRelBase.expand(BeamSetOperatorRelBase.java:96) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSetOperatorRelBase.expand(BeamSetOperatorRelBase.java:41) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39) > at > cloud.dataflow.sql.ExecuteQueryServiceServer$SqlComplianceServiceImpl.executeQuery(ExecuteQueryServiceServer.java:288) > at > com.google.zetasql.testing.SqlComplianceServiceGrpc$MethodHandlers.invoke(SqlComplianceServiceGrpc.java:423) > at > com.google.zetasql.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171) > at > com.google.zetasql.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283) > at > com.google.zetasql.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:711) > at > com.google.zetasql.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) > at > com.google.zetasql.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > {code} > 1: > {code} > Apr 01, 2020 3:55:55 PM > cloud.dataflow.sql.ExecuteQueryServiceServer$SqlComplianceServiceImpl > executeQuery > INFO: Processing Sql statement: SELECT double_val FROM Table1 UNION ALL > SELECT double_val FROM Table2 > Apr 01, 2020 3:55:55 PM > cloud.dataflow.sql.ExecuteQueryServiceServer$SqlComplianceServiceImpl > executeQuery > SEVERE: !!!!!the keyCoder of a GroupByKey must be deterministic > java.lang.IllegalStateException: the keyCoder of a GroupByKey must be > deterministic > at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:234) > at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:110) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493) > at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:368) > at > org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:118) > at > org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:71) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:476) > at > org.apache.beam.sdk.transforms.join.KeyedPCollectionTuple.apply(KeyedPCollectionTuple.java:108) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSetOperatorRelBase.expand(BeamSetOperatorRelBase.java:96) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSetOperatorRelBase.expand(BeamSetOperatorRelBase.java:41) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39) > at > cloud.dataflow.sql.ExecuteQueryServiceServer$SqlComplianceServiceImpl.executeQuery(ExecuteQueryServiceServer.java:288) > at > com.google.zetasql.testing.SqlComplianceServiceGrpc$MethodHandlers.invoke(SqlComplianceServiceGrpc.java:423) > at > com.google.zetasql.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171) > at > com.google.zetasql.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283) > at > com.google.zetasql.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:711) > at > com.google.zetasql.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) > at > com.google.zetasql.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.beam.sdk.coders.Coder$NonDeterministicException: > SchemaCoder<Schema: Fields: > Field{name=double_val, description=, type=FieldType{typeName=DOUBLE, > nullable=true, logicalType=null, collectionElementType=null, mapKeyType=null, > mapValueType=null, rowSchema=null, metadata={}}, options={{}}} > Options:{{}} UUID: 9896a9ff-6b85-423e-bbd0-392f0461e87b delegateCoder: > org.apache.beam.sdk.coders.Coder$ByteBuddy$lYxrgbBO@b517475 is not > deterministic because: > All fields must have deterministic encoding > at org.apache.beam.sdk.coders.Coder.verifyDeterministic(Coder.java:196) > at > org.apache.beam.sdk.schemas.SchemaCoder.verifyDeterministic(SchemaCoder.java:138) > at > org.apache.beam.sdk.schemas.SchemaCoder.verifyDeterministic(SchemaCoder.java:126) > at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:232) > ... 25 more > Caused by: org.apache.beam.sdk.coders.Coder$NonDeterministicException: > NullableCoder(DoubleCoder) is not deterministic because: > Value coder must be deterministic > at org.apache.beam.sdk.coders.Coder.verifyDeterministic(Coder.java:196) > at org.apache.beam.sdk.coders.Coder.verifyDeterministic(Coder.java:207) > at > org.apache.beam.sdk.coders.NullableCoder.verifyDeterministic(NullableCoder.java:109) > at org.apache.beam.sdk.coders.Coder.verifyDeterministic(Coder.java:194) > ... 28 more > Caused by: org.apache.beam.sdk.coders.Coder$NonDeterministicException: > DoubleCoder is not deterministic because: > Floating point encodings are not guaranteed to be deterministic. > at > org.apache.beam.sdk.coders.DoubleCoder.verifyDeterministic(DoubleCoder.java:71) > at org.apache.beam.sdk.coders.Coder.verifyDeterministic(Coder.java:194) > ... 31 more > {code} > 2: > {code} > Apr 01, 2020 3:55:59 PM > cloud.dataflow.sql.ExecuteQueryServiceServer$SqlComplianceServiceImpl > executeQuery > INFO: Processing Sql statement: SELECT bool_val, double_val, int64_val, > str_val FROM Table1 > UNION ALL > SELECT bool_val, double_val, int64_val, str_val FROM Table2 > Apr 01, 2020 3:55:59 PM > cloud.dataflow.sql.ExecuteQueryServiceServer$SqlComplianceServiceImpl > executeQuery > SEVERE: !!!!!the keyCoder of a GroupByKey must be deterministic > java.lang.IllegalStateException: the keyCoder of a GroupByKey must be > deterministic > at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:234) > at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:110) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493) > at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:368) > at > org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:118) > at > org.apache.beam.sdk.transforms.join.CoGroupByKey.expand(CoGroupByKey.java:71) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:476) > at > org.apache.beam.sdk.transforms.join.KeyedPCollectionTuple.apply(KeyedPCollectionTuple.java:108) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSetOperatorRelBase.expand(BeamSetOperatorRelBase.java:96) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSetOperatorRelBase.expand(BeamSetOperatorRelBase.java:41) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39) > at > cloud.dataflow.sql.ExecuteQueryServiceServer$SqlComplianceServiceImpl.executeQuery(ExecuteQueryServiceServer.java:288) > at > com.google.zetasql.testing.SqlComplianceServiceGrpc$MethodHandlers.invoke(SqlComplianceServiceGrpc.java:423) > at > com.google.zetasql.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171) > at > com.google.zetasql.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283) > at > com.google.zetasql.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:711) > at > com.google.zetasql.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) > at > com.google.zetasql.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.beam.sdk.coders.Coder$NonDeterministicException: > SchemaCoder<Schema: Fields: > Field{name=bool_val, description=, type=FieldType{typeName=BOOLEAN, > nullable=true, logicalType=null, collectionElementType=null, mapKeyType=null, > mapValueType=null, rowSchema=null, metadata={}}, options={{}}} > Field{name=double_val, description=, type=FieldType{typeName=DOUBLE, > nullable=true, logicalType=null, collectionElementType=null, mapKeyType=null, > mapValueType=null, rowSchema=null, metadata={}}, options={{}}} > Field{name=int64_val, description=, type=FieldType{typeName=INT64, > nullable=true, logicalType=null, collectionElementType=null, mapKeyType=null, > mapValueType=null, rowSchema=null, metadata={}}, options={{}}} > Field{name=str_val, description=, type=FieldType{typeName=STRING, > nullable=true, logicalType=null, collectionElementType=null, mapKeyType=null, > mapValueType=null, rowSchema=null, metadata={}}, options={{}}} > Options:{{}} UUID: dc5cfc6f-96bd-4181-8f13-a9186fc8cdf9 delegateCoder: > org.apache.beam.sdk.coders.Coder$ByteBuddy$tPVlD9JI@7c4b1367 is not > deterministic because: > All fields must have deterministic encoding > at org.apache.beam.sdk.coders.Coder.verifyDeterministic(Coder.java:196) > at > org.apache.beam.sdk.schemas.SchemaCoder.verifyDeterministic(SchemaCoder.java:138) > at > org.apache.beam.sdk.schemas.SchemaCoder.verifyDeterministic(SchemaCoder.java:126) > at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:232) > ... 25 more > Caused by: org.apache.beam.sdk.coders.Coder$NonDeterministicException: > NullableCoder(DoubleCoder) is not deterministic because: > Value coder must be deterministic > at org.apache.beam.sdk.coders.Coder.verifyDeterministic(Coder.java:196) > at org.apache.beam.sdk.coders.Coder.verifyDeterministic(Coder.java:207) > at > org.apache.beam.sdk.coders.NullableCoder.verifyDeterministic(NullableCoder.java:109) > at org.apache.beam.sdk.coders.Coder.verifyDeterministic(Coder.java:194) > ... 28 more > Caused by: org.apache.beam.sdk.coders.Coder$NonDeterministicException: > DoubleCoder is not deterministic because: > Floating point encodings are not guaranteed to be deterministic. > at > org.apache.beam.sdk.coders.DoubleCoder.verifyDeterministic(DoubleCoder.java:71) > at org.apache.beam.sdk.coders.Coder.verifyDeterministic(Coder.java:194) > ... 31 more > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)