Re: UDTF registration fails for hiveEnabled SQLContext

[Mick Davies]

I am trying to register a UDTF not a UDF.

So I don't think this applies

Mick



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: UDTF registration fails for hiveEnabled SQLContext

[Ajay]

You can register udf's by using the in-built udf function as well using (import
org.apache.spark.sql.functions._)
Something along the lines of
val flattenUdf = udf(udfutils.flatten)
where udfutils is another object and flatten is a method in it.

On Tue, May 15, 2018 at 3:27 AM Mick Davies 
wrote:

> Hi Gourav,
>
> I don't think you can register UDTFs via sparkSession.udf.register
>
> Mick
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>

-- 
Thanks,
Ajay

Re: UDTF registration fails for hiveEnabled SQLContext

[Mick Davies]

Hi Gourav,

I don't think you can register UDTFs via sparkSession.udf.register

Mick



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: UDTF registration fails for hiveEnabled SQLContext

[Gourav Sengupta]

Hi Mick,

the error practically gives you the suggestion to correct it "use
sparkSession.udf.register(...)
instead".

Regards,
Gourav Sengupta

On Mon, May 14, 2018 at 8:59 AM, Mick Davies 
wrote:

> The examples were lost by formatting:
>
> Exception is:
>
> No handler for UDAF 'com.iqvia.rwas.omop.udtf.ParallelExplode'. Use
> sparkSession.udf.register(...) instead.; line 1 pos 7
> org.apache.spark.sql.AnalysisException: No handler for UDAF
> 'com.iqvia.rwas.omop.udtf.ParallelExplode'. Use
> sparkSession.udf.register(...) instead.; line 1 pos 7
> at
> org.apache.spark.sql.catalyst.catalog.SessionCatalog.
> makeFunctionExpression(SessionCatalog.scala:1134)
> at
> org.apache.spark.sql.catalyst.catalog.SessionCatalog$$
> anonfun$org$apache$spark$sql$catalyst$catalog$SessionCatalog$$
> makeFunctionBuilder$1.apply(SessionCatalog.scala:1114)
> at
> org.apache.spark.sql.catalyst.catalog.SessionCatalog$$
> anonfun$org$apache$spark$sql$catalyst$catalog$SessionCatalog$$
> makeFunctionBuilder$1.apply(SessionCatalog.scala:1114)
> at
> org.apache.spark.sql.catalyst.analysis.SimpleFunctionRegistry.
> lookupFunction(FunctionRegistry.scala:115)
> at
> org.apache.spark.sql.catalyst.catalog.SessionCatalog.
> lookupFunction(SessionCatalog.scala:1245)
>
>
>
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>

Re: UDTF registration fails for hiveEnabled SQLContext

[Mick Davies]

The examples were lost by formatting:

Exception is:

No handler for UDAF 'com.iqvia.rwas.omop.udtf.ParallelExplode'. Use
sparkSession.udf.register(...) instead.; line 1 pos 7
org.apache.spark.sql.AnalysisException: No handler for UDAF
'com.iqvia.rwas.omop.udtf.ParallelExplode'. Use
sparkSession.udf.register(...) instead.; line 1 pos 7
at
org.apache.spark.sql.catalyst.catalog.SessionCatalog.makeFunctionExpression(SessionCatalog.scala:1134)
at
org.apache.spark.sql.catalyst.catalog.SessionCatalog$$anonfun$org$apache$spark$sql$catalyst$catalog$SessionCatalog$$makeFunctionBuilder$1.apply(SessionCatalog.scala:1114)
at
org.apache.spark.sql.catalyst.catalog.SessionCatalog$$anonfun$org$apache$spark$sql$catalyst$catalog$SessionCatalog$$makeFunctionBuilder$1.apply(SessionCatalog.scala:1114)
at
org.apache.spark.sql.catalyst.analysis.SimpleFunctionRegistry.lookupFunction(FunctionRegistry.scala:115)
at
org.apache.spark.sql.catalyst.catalog.SessionCatalog.lookupFunction(SessionCatalog.scala:1245)






--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org