Yang Zhang created BEAM-8316:
--------------------------------
Summary: What is corresponding data type to set in UDF parameter
to match RecordType
Key: BEAM-8316
URL: https://issues.apache.org/jira/browse/BEAM-8316
Project: Beam
Issue Type: Bug
Components: beam-model
Affects Versions: 2.15.0
Reporter: Yang Zhang
Hello Beam community,
I want to have an UDF to take a record as input. Per error info as shown below,
it indicates that the input is *RecordType*, but ** what should I set in the
UDF parameter so that Beam would not complain about the type compatibility?
Below is the rull error trace. Thank you very much!
========error trace===========
Exception in thread "main"
org.apache.beam.sdk.extensions.sql.impl.ParseException: Unable to parse query
select fooudf(pv.header) from kafka.tracking.PageViewEvent as pvException in
thread "main" org.apache.beam.sdk.extensions.sql.impl.ParseException: Unable to
parse query select fooudf(pv.header) from kafka.tracking.PageViewEvent as pv at
org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.convertToBeamRel(CalciteQueryPlanner.java:165)
at
org.apache.beam.sdk.extensions.sql.impl.BeamSqlEnv.parseQuery(BeamSqlEnv.java:103)
at
org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:124)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:82)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:539) at
org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:473) at
org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44) at
org.apache.beam.sdk.Pipeline.apply(Pipeline.java:169) at
com.linkedin.samza.sql.engine.BeamSqlEntry.preparePipeline(BeamSqlEntry.java:52)
at com.linkedin.samza.sql.engine.BeamSqlEntry.exec(BeamSqlEntry.java:41) at
com.linkedin.samza.sql.engine.BeamSqlUI.main(BeamSqlUI.java:33)Caused by:
org.apache.calcite.tools.ValidationException:
org.apache.calcite.runtime.CalciteContextException: >From line 1, column 8 to
line 1, column 24: No match found for function signature
fooudf(<RecordType(INTEGER memberId, VARCHAR viewerUrn, VARCHAR
applicationViewerUrn, VARCHAR csUserUrn, BIGINT time, VARCHAR server, VARCHAR
service, VARCHAR environment, VARBINARY guid, VARBINARY treeId, INTEGER
requestId, VARCHAR impersonatorId, VARCHAR version, VARCHAR instance, VARCHAR
appName, VARCHAR testId, VARCHAR testSegmentId, RecordType(BIGINT time, VARCHAR
server, VARCHAR instance, VARCHAR appName, VARBINARY messageId, INTEGER
auditVersion, VARCHAR fabricUrn, VARCHAR clusterConnectionString) auditHeader,
RecordType(VARCHAR pageUrn, VARBINARY trackingId) pageInstance,
RecordType(VARCHAR applicationUrn, VARCHAR version, VARBINARY trackingId)
clientApplicationInstance, VARCHAR originSource, VARCHAR sessionUrn,
RecordType(VARBINARY treeId, INTEGER requestId, INTEGER taskId, VARCHAR
rpcTrace, BOOLEAN forceTraceEnabled, (VARCHAR, VARCHAR) MAP context, DOUBLE
scaleFactor) traceData)>) at
org.apache.calcite.prepare.PlannerImpl.validate(PlannerImpl.java:190) at
org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.convertToBeamRel(CalciteQueryPlanner.java:136)
... 10 moreCaused by: org.apache.calcite.runtime.CalciteContextException: From
line 1, column 8 to line 1, column 24: No match found for function signature
fooudf(<RecordType(INTEGER memberId, VARCHAR viewerUrn, VARCHAR
applicationViewerUrn, VARCHAR csUserUrn, BIGINT time, VARCHAR server, VARCHAR
service, VARCHAR environment, VARBINARY guid, VARBINARY treeId, INTEGER
requestId, VARCHAR impersonatorId, VARCHAR version, VARCHAR instance, VARCHAR
appName, VARCHAR testId, VARCHAR testSegmentId, RecordType(BIGINT time, VARCHAR
server, VARCHAR instance, VARCHAR appName, VARBINARY messageId, INTEGER
auditVersion, VARCHAR fabricUrn, VARCHAR clusterConnectionString) auditHeader,
RecordType(VARCHAR pageUrn, VARBINARY trackingId) pageInstance,
RecordType(VARCHAR applicationUrn, VARCHAR version, VARBINARY trackingId)
clientApplicationInstance, VARCHAR originSource, VARCHAR sessionUrn,
RecordType(VARBINARY treeId, INTEGER requestId, INTEGER taskId, VARCHAR
rpcTrace, BOOLEAN forceTraceEnabled, (VARCHAR, VARCHAR) MAP context, DOUBLE
scaleFactor) traceData)>) at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) at
org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:787) at
org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:772) at
org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4825)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1739)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:270) at
org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:215) at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5584)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5571)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138) at
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1657)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1642)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:462)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4089)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3352)
at
org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:994)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:954)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216) at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:929)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:633)
at org.apache.calcite.prepare.PlannerImpl.validate(PlannerImpl.java:188) ...
11 moreCaused by: org.apache.calcite.sql.validate.SqlValidatorException: No
match found for function signature fooudf(<RecordType(INTEGER memberId, VARCHAR
viewerUrn, VARCHAR applicationViewerUrn, VARCHAR csUserUrn, BIGINT time,
VARCHAR server, VARCHAR service, VARCHAR environment, VARBINARY guid, VARBINARY
treeId, INTEGER requestId, VARCHAR impersonatorId, VARCHAR version, VARCHAR
instance, VARCHAR appName, VARCHAR testId, VARCHAR testSegmentId,
RecordType(BIGINT time, VARCHAR server, VARCHAR instance, VARCHAR appName,
VARBINARY messageId, INTEGER auditVersion, VARCHAR fabricUrn, VARCHAR
clusterConnectionString) auditHeader, RecordType(VARCHAR pageUrn, VARBINARY
trackingId) pageInstance, RecordType(VARCHAR applicationUrn, VARCHAR version,
VARBINARY trackingId) clientApplicationInstance, VARCHAR originSource, VARCHAR
sessionUrn, RecordType(VARBINARY treeId, INTEGER requestId, INTEGER taskId,
VARCHAR rpcTrace, BOOLEAN forceTraceEnabled, (VARCHAR, VARCHAR) MAP context,
DOUBLE scaleFactor) traceData)>) at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) at
org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572) ... 33 more
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
