Hi Sonam,
org.apache.beam.sdk.extensions.sql.zetasql.translation.SqlOperators$1
> cannot be cast to
> org.apache.beam.vendor.calcite.v1_20_0.org.apache.calcite.sql.SqlAggFunction
> java.lang.ClassCastException:
> org.apache.beam.sdk.extensions.sql.zetasql.translation.SqlOperators$1
> cannot be cast to
> org.apache.beam.vendor.calcite.v1_20_0.org.apache.calcite.sql.SqlAggFunction
> at
> org.apache.beam.sdk.extensions.sql.zetasql.translation.AggregateScanConverter.convertAggCall(AggregateScanConverter.java:202)
>
If you go to line 202 of `AggregateScanConverter` you will find
SqlAggFunction sqlAggFunction =
> (SqlAggFunction)
>
> SqlOperatorMappingTable.ZETASQL_FUNCTION_TO_CALCITE_SQL_OPERATOR.get(
> aggregateFunctionCall.getFunction().getName());
>
where the result got from the map is being casted to SqlAggFunction. The
element created here in your file SqlOperators.java line 137:
public static final SqlOperator LOGICAL_AND =
> createUdfOperator("LOGICAL_AND", BeamBuiltinMethods.LOGICAL_AND);
>
createUdfOperator() returns a SqlUserDefinedFunction which is not
a SqlAggFunction; I believe you should use createUdafOperator() instead,
which returns a SqlUserDefinedAggFunction, a subclass of SqlAggFunction.
See STRING_AGG_STRING_FN as an example.
Best,
Robin
On Tue, Nov 17, 2020 at 1:36 AM Sonam Ramchand <
sonam.ramch...@venturedive.com> wrote:
>
> Hi Devs,
> Following the guidelines, I implemented CmobineFn Logical_AND as
> https://github.com/sonam-vend/beam/commit/9ad8ee1d8fa617aca7fcafc8e7efe8bf388b3afb.
> But i am getting
>
>
> *org.apache.beam.sdk.extensions.sql.zetasql.translation.SqlOperators$1
> cannot be cast to
> org.apache.beam.vendor.calcite.v1_20_0.org.apache.calcite.sql.SqlAggFunction*
> java.lang.ClassCastException:
> *org.apache.beam.sdk.extensions.sql.zetasql.translation.SqlOperators$1
> cannot be cast to
> org.apache.beam.vendor.calcite.v1_20_0.org.apache.calcite.sql.SqlAggFunction*
> at
> org.apache.beam.sdk.extensions.sql.zetasql.translation.AggregateScanConverter.convertAggCall(AggregateScanConverter.java:202)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.translation.AggregateScanConverter.convert(AggregateScanConverter.java:94)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.translation.AggregateScanConverter.convert(AggregateScanConverter.java:50)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.translation.QueryStatementConverter.convertNode(QueryStatementConverter.java:99)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at java.util.Collections$2.tryAdvance(Collections.java:4719)
> at java.util.Collections$2.forEachRemaining(Collections.java:4727)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.translation.QueryStatementConverter.convertNode(QueryStatementConverter.java:98)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.translation.QueryStatementConverter.convert(QueryStatementConverter.java:86)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.translation.QueryStatementConverter.convertRootQuery(QueryStatementConverter.java:52)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLPlannerImpl.rel(ZetaSQLPlannerImpl.java:140)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner.convertToBeamRelInternal(ZetaSQLQueryPlanner.java:168)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner.convertToBeamRel(ZetaSQLQueryPlanner.java:156)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner.convertToBeamRel(ZetaSQLQueryPlanner.java:140)
> at
> org.apache.beam.sdk.extensions.sql.zetasql.ZetaSqlDialectSpecTest.testLogicalAndZetaSQL(ZetaSqlDialectSpecTest.java:4334)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319)
> at
> org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:266)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:365)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:330)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:78)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:328)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:65)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:412)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
> at
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
> at
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
> at
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
> at
> org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
> at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> at
> org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
> at
> org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
> at
> org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
> at
> org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
> at
> org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
> at java.lang.Thread.run(Thread.java:748)
>
> org.apache.beam.sdk.extensions.sql.zetasql.ZetaSqlDialectSpecTest >
> testLogicalAndZetaSQL FAILED
> java.lang.ClassCastException at ZetaSqlDialectSpecTest.java:4334
> 1 test completed, 1 failed
> > Task :sdks:java:extensions:sql:zetasql:test FAILED
> FAILURE: Build failed with an exception.
> * What went wrong:
> Execution failed for task ':sdks:java:extensions:sql:zetasql:test'.
> > There were failing tests. See the report at:
> file:///home/vend/ApacheBeam/beam/sdks/java/extensions/sql/zetasql/build/reports/tests/test/index.html
>
> Do you have any idea why this is happening?
>
>
> On Fri, Nov 13, 2020 at 11:42 PM Robin Qiu <robi...@google.com> wrote:
>
>> Oh I see. Thanks for the clarification, Kenn! Yeah the CombineFn
>> LOGICAL_AND is to be implemented.
>>
>> On Fri, Nov 13, 2020 at 10:00 AM Kenneth Knowles <k...@apache.org> wrote:
>>
>>> Some clarification: LOGICAL_AND is a ZetaSQL/BigQuery aggregate
>>> function:
>>> https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#logical_and
>>>
>>> So it needs to be implemented as a CombineFn. Here are some example PRs
>>> that do similar things: https://github.com/apache/beam/pulls?q=BIT_OR
>>>
>>> Kenn
>>>
>>> On Thu, Nov 12, 2020 at 12:48 PM Rui Wang <ruw...@google.com> wrote:
>>>
>>>> Or the question is, which BeamSQL dialect you are using?
>>>>
>>>>
>>>> -Rui
>>>>
>>>> On Thu, Nov 12, 2020 at 12:41 PM Robin Qiu <robi...@google.com> wrote:
>>>>
>>>>> Hi Sonam, AND operator is already defined by Calcite and the mapping
>>>>> is here:
>>>>> https://github.com/apache/beam/blob/816017e44e3209d334f4f3b2bc3fa829663c530e/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperatorMappingTable.java#L39.
>>>>> Is this what you are looking for?
>>>>>
>>>>> On Thu, Nov 12, 2020 at 12:00 PM Kyle Weaver <kcwea...@google.com>
>>>>> wrote:
>>>>>
>>>>>> If you're defining a new built-in function in ZetaSQL, you can define
>>>>>> an operator for it here:
>>>>>> https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
>>>>>>
>>>>>> Then add the operator the table here:
>>>>>> https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperatorMappingTable.java
>>>>>>
>>>>>> On Thu, Nov 12, 2020 at 11:14 AM Sonam Ramchand <
>>>>>> sonam.ramch...@venturedive.com> wrote:
>>>>>>
>>>>>>> There is no LOGICAL_AND operator in SqlStdOperatorTable, is there
>>>>>>> any other way to implement LOGICAL_AND?
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Regards,
>>>>>>> *Sonam*
>>>>>>> Software Engineer
>>>>>>> Mobile: +92 3088337296 <+92%20308%208337296>
>>>>>>>
>>>>>>> <http://venturedive.com/>
>>>>>>>
>>>>>>
>
> --
>
> Regards,
> *Sonam*
> Software Engineer
> Mobile: +92 3088337296 <+92%20308%208337296>
>
> <http://venturedive.com/>
>
