GitHub user WeichenXu123 opened a pull request:
https://github.com/apache/spark/pull/20934
[SPARK-23818][SQL][WIP] an official UDF interface for Spark SQL
## What changes were proposed in this pull request?
API: (to be discussed), use 2-args as example:
```
UDFRegistration
// scala version
def registerV2(name: String, func: Function2[_, _, _], returnType:
DataType,
input1Type: DataType, input2Type: DataType, nullable: Boolean,
deterministic: Boolean):UserDefinedFunctionV2
// java version
def registerV2j(name: String, f: UDF2[_, _, _], returnType: DataType,
input1Type: DataType, input2Type: DataType, nullable: Boolean,
deterministic: Boolean):UserDefinedFunctionV2
```
## How was this patch tested?
Two simple testcase added.
Others will add later.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/WeichenXu123/spark udfv2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20934.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20934
----
commit 2ef96133982eb1574369867f8dfb80269fabf079
Author: WeichenXu <weichen.xu@...>
Date: 2018-03-28T11:57:52Z
init pr
commit a1439e121f43e7025bd63df31055a2c871abb6ec
Author: WeichenXu <weichen.xu@...>
Date: 2018-03-29T05:12:17Z
add udf registration & scala test
commit d582675bf407e03016db9ac4955f36fbdd955588
Author: WeichenXu <weichen.xu@...>
Date: 2018-03-29T10:35:06Z
fix several bugs
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
