Samuel Waggoner created BEAM-3739:
-------------------------------------
Summary: @Parameter annotation does not work for UDFs in Beam SQL
Key: BEAM-3739
URL: https://issues.apache.org/jira/browse/BEAM-3739
Project: Beam
Issue Type: Bug
Components: dsl-sql
Affects Versions: 2.3.0
Reporter: Samuel Waggoner
Assignee: Xu Mingmin
BeamSqlUdf javadoc indicates you can have optional parameters, but this
functionality is not working. I implemented the following copy/paste from the
doc
https://beam.apache.org/documentation/sdks/javadoc/2.3.0/org/apache/beam/sdk/extensions/sql/BeamSqlUdf.html:
{code:java}
public static class MyLeftFunction implements BeamSqlUdf {
public String eval(
@Parameter(name = "s") String s,
@Parameter(name = "n", optional = true) Integer n) {
return s.substring(0, n == null ? 1 : n);
}
}{code}
I modify a query in BeamSqlExample.java to use it. With all parameters
supplied, it completes successfully:
{code:java}
//Case 1. run a simple SQL query over input PCollection with
BeamSql.simpleQuery;
PCollection<Row> outputStream = inputTable.apply(
BeamSql.query("select c1, leftfn('string1', 1) as c2, c3 from PCOLLECTION
where c1 > 1")
.registerUdf("leftfn", MyLeftFunction.class));{code}
With the optional parameter left off, I get an exception:
{code:java}
//Case 1. run a simple SQL query over input PCollection with
BeamSql.simpleQuery;
PCollection<Row> outputStream = inputTable.apply(
BeamSql.query("select c1, leftfn('string1') as c2, c3 from PCOLLECTION where
c1 > 1")
.registerUdf("leftfn", MyLeftFunction.class));{code}
{code:java}
Exception in thread "main" java.lang.IllegalStateException:
java.lang.UnsupportedOperationException: Operator: DEFAULT is not supported yet!
at
org.apache.beam.sdk.extensions.sql.QueryTransform.expand(QueryTransform.java:75)
at
org.apache.beam.sdk.extensions.sql.QueryTransform.expand(QueryTransform.java:47)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:472)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:286)
at
org.apache.beam.sdk.extensions.sql.example.BeamSqlExample.main(BeamSqlExample.java:76)
Caused by: java.lang.UnsupportedOperationException: Operator: DEFAULT is not
supported yet!
at
org.apache.beam.sdk.extensions.sql.impl.interpreter.BeamSqlFnExecutor.buildExpression(BeamSqlFnExecutor.java:424)
at
org.apache.beam.sdk.extensions.sql.impl.interpreter.BeamSqlFnExecutor.buildExpression(BeamSqlFnExecutor.java:201)
at
org.apache.beam.sdk.extensions.sql.impl.interpreter.BeamSqlFnExecutor.<init>(BeamSqlFnExecutor.java:125)
at
org.apache.beam.sdk.extensions.sql.impl.rel.BeamProjectRel.buildBeamPipeline(BeamProjectRel.java:70)
at
org.apache.beam.sdk.extensions.sql.QueryTransform.expand(QueryTransform.java:73)
... 5 more{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
