Hi Saif,
There are 2 types of UDFs. Those used by SQL and those used by the Scala DSL.
For SQL, you just register a function like so (this example is from the docs):
sqlContext.udf.register(“strLen”, (s: String) => s.length)
sqlContext.sql(“select name, strLen(name) from people”).show
The other method, for Scala DSL, instead:
import org.apache.spark.sql.functions._
def strLen = udf { (s: String) => s.length }
people.select(people(“name”), strLen(people(“name”))).show
Thanks,
Silvio
From: "saif.a.ell...@wellsfargo.com<mailto:saif.a.ell...@wellsfargo.com>"
Date: Monday, September 14, 2015 at 12:39 PM
To: "user@spark.apache.org<mailto:user@spark.apache.org>"
Subject: Where can I learn how to write udf?
Hi all,
I am failing to find a proper guide or tutorial onto how to write proper udf
functions in scala.
Appreciate the effort saving,
Saif
