[
https://issues.apache.org/jira/browse/BEAM-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17548285#comment-17548285
]
Danny McCormick commented on BEAM-8316:
---------------------------------------
This issue has been migrated to https://github.com/apache/beam/issues/19927
> What is corresponding data type to set in UDF parameter to match RecordType
> ---------------------------------------------------------------------------
>
> Key: BEAM-8316
> URL: https://issues.apache.org/jira/browse/BEAM-8316
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql
> Affects Versions: 2.15.0
> Reporter: Yang Zhang
> Priority: P3
>
> Hello Beam community,
> I want to have an UDF to take a record as input. Per error info as shown
> below, it indicates that the input is *RecordType*, but ** what should I set
> in the UDF parameter so that Beam would not complain about the type
> compatibility? Below is the rull error trace. Thank you very much!
>
> ========error trace===========
> Exception in thread "main"
> org.apache.beam.sdk.extensions.sql.impl.ParseException: Unable to parse query
> select fooudf(pv.header) from kafka.tracking.PageViewEvent as pvException in
> thread "main" org.apache.beam.sdk.extensions.sql.impl.ParseException: Unable
> to parse query select fooudf(pv.header) from kafka.tracking.PageViewEvent as
> pv at
> org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.convertToBeamRel(CalciteQueryPlanner.java:165)
> at
> org.apache.beam.sdk.extensions.sql.impl.BeamSqlEnv.parseQuery(BeamSqlEnv.java:103)
> at
> org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:124)
> at
> org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:82)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:539) at
> org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:473) at
> org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44) at
> org.apache.beam.sdk.Pipeline.apply(Pipeline.java:169) at
> com.linkedin.samza.sql.engine.BeamSqlEntry.preparePipeline(BeamSqlEntry.java:52)
> at com.linkedin.samza.sql.engine.BeamSqlEntry.exec(BeamSqlEntry.java:41) at
> com.linkedin.samza.sql.engine.BeamSqlUI.main(BeamSqlUI.java:33)Caused by:
> org.apache.calcite.tools.ValidationException:
> org.apache.calcite.runtime.CalciteContextException: From line 1, column 8 to
> line 1, column 24: No match found for function signature
> fooudf(<RecordType(INTEGER memberId, VARCHAR viewerUrn, VARCHAR
> applicationViewerUrn, VARCHAR csUserUrn, BIGINT time, VARCHAR server, VARCHAR
> service, VARCHAR environment, VARBINARY guid, VARBINARY treeId, INTEGER
> requestId, VARCHAR impersonatorId, VARCHAR version, VARCHAR instance, VARCHAR
> appName, VARCHAR testId, VARCHAR testSegmentId, RecordType(BIGINT time,
> VARCHAR server, VARCHAR instance, VARCHAR appName, VARBINARY messageId,
> INTEGER auditVersion, VARCHAR fabricUrn, VARCHAR clusterConnectionString)
> auditHeader, RecordType(VARCHAR pageUrn, VARBINARY trackingId) pageInstance,
> RecordType(VARCHAR applicationUrn, VARCHAR version, VARBINARY trackingId)
> clientApplicationInstance, VARCHAR originSource, VARCHAR sessionUrn,
> RecordType(VARBINARY treeId, INTEGER requestId, INTEGER taskId, VARCHAR
> rpcTrace, BOOLEAN forceTraceEnabled, (VARCHAR, VARCHAR) MAP context, DOUBLE
> scaleFactor) traceData)>) at
> org.apache.calcite.prepare.PlannerImpl.validate(PlannerImpl.java:190) at
> org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.convertToBeamRel(CalciteQueryPlanner.java:136)
> ... 10 moreCaused by: org.apache.calcite.runtime.CalciteContextException:
> From line 1, column 8 to line 1, column 24: No match found for function
> signature fooudf(<RecordType(INTEGER memberId, VARCHAR viewerUrn, VARCHAR
> applicationViewerUrn, VARCHAR csUserUrn, BIGINT time, VARCHAR server, VARCHAR
> service, VARCHAR environment, VARBINARY guid, VARBINARY treeId, INTEGER
> requestId, VARCHAR impersonatorId, VARCHAR version, VARCHAR instance, VARCHAR
> appName, VARCHAR testId, VARCHAR testSegmentId, RecordType(BIGINT time,
> VARCHAR server, VARCHAR instance, VARCHAR appName, VARBINARY messageId,
> INTEGER auditVersion, VARCHAR fabricUrn, VARCHAR clusterConnectionString)
> auditHeader, RecordType(VARCHAR pageUrn, VARBINARY trackingId) pageInstance,
> RecordType(VARCHAR applicationUrn, VARCHAR version, VARBINARY trackingId)
> clientApplicationInstance, VARCHAR originSource, VARCHAR sessionUrn,
> RecordType(VARBINARY treeId, INTEGER requestId, INTEGER taskId, VARCHAR
> rpcTrace, BOOLEAN forceTraceEnabled, (VARCHAR, VARCHAR) MAP context, DOUBLE
> scaleFactor) traceData)>) at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
> org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
> at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:787) at
> org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:772) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4825)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1739)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:270) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:215) at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5584)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5571)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1657)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1642)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:462)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4089)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3352)
> at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
> at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:994)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:954)
> at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:929)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:633)
> at org.apache.calcite.prepare.PlannerImpl.validate(PlannerImpl.java:188) ...
> 11 moreCaused by: org.apache.calcite.sql.validate.SqlValidatorException: No
> match found for function signature fooudf(<RecordType(INTEGER memberId,
> VARCHAR viewerUrn, VARCHAR applicationViewerUrn, VARCHAR csUserUrn, BIGINT
> time, VARCHAR server, VARCHAR service, VARCHAR environment, VARBINARY guid,
> VARBINARY treeId, INTEGER requestId, VARCHAR impersonatorId, VARCHAR version,
> VARCHAR instance, VARCHAR appName, VARCHAR testId, VARCHAR testSegmentId,
> RecordType(BIGINT time, VARCHAR server, VARCHAR instance, VARCHAR appName,
> VARBINARY messageId, INTEGER auditVersion, VARCHAR fabricUrn, VARCHAR
> clusterConnectionString) auditHeader, RecordType(VARCHAR pageUrn, VARBINARY
> trackingId) pageInstance, RecordType(VARCHAR applicationUrn, VARCHAR version,
> VARBINARY trackingId) clientApplicationInstance, VARCHAR originSource,
> VARCHAR sessionUrn, RecordType(VARBINARY treeId, INTEGER requestId, INTEGER
> taskId, VARCHAR rpcTrace, BOOLEAN forceTraceEnabled, (VARCHAR, VARCHAR) MAP
> context, DOUBLE scaleFactor) traceData)>) at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
> org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
> at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572) ... 33
> more
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
