GitHub user gatorsmile reopened a pull request: https://github.com/apache/spark/pull/20171
[SPARK-22978] [PySpark] Register Vectorized UDFs for SQL Statement [WIP] ## What changes were proposed in this pull request? Register Vectorized UDFs for SQL Statement. For example, ```Python >>> import random >>> from pyspark.sql.types import IntegerType >>> from pyspark.sql.functions import pandas_udf >>> random_pandas_udf = pandas_udf( ... lambda x: random.randint(0, 100) + x, IntegerType()) ... .asNondeterministic() # doctest: +SKIP >>> _ = spark.catalog.registerFunction( ... "random_pandas_udf", random_pandas_udf, IntegerType()) # doctest: +SKIP >>> spark.sql("SELECT random_pandas_udf(2)").collect() # doctest: +SKIP [Row(random_pandas_udf(2)=84)] ``` ## How was this patch tested? Added test cases You can merge this pull request into a Git repository by running: $ git pull https://github.com/gatorsmile/spark supportVectorizedUDF Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20171.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20171 ---- commit 5e0c8e1f08b4bb0716b8f21a8393acea361981dc Author: gatorsmile <gatorsmile@...> Date: 2018-01-06T08:06:36Z fix commit f41e74e9ba87900dbd66cec12fbb579755d4290b Author: gatorsmile <gatorsmile@...> Date: 2018-01-06T08:20:41Z Merge remote-tracking branch 'upstream/master' into supportVectorizedUDF commit 3983bcbbd7a1279b899e72d07822afa5d4ef7749 Author: gatorsmile <gatorsmile@...> Date: 2018-01-06T08:36:37Z rename commit fe8dcbe47c7800e771420b416f6e5352c745e85c Author: gatorsmile <gatorsmile@...> Date: 2018-01-06T09:09:44Z fix commit 3411dccc6ab57f3135a99d2b2b535fd5c135cbc7 Author: gatorsmile <gatorsmile@...> Date: 2018-01-06T10:00:05Z import commit b801e7094c7c730adb53a25e13363973930a0b42 Author: gatorsmile <gatorsmile@...> Date: 2018-01-06T11:41:33Z fix commit 3c08f3d6b7ec58735260de687bb74b104e6f7009 Author: gatorsmile <gatorsmile@...> Date: 2018-01-07T00:41:22Z fix commit 423c832b89f8bef5f4812d24c67921644f169f15 Author: gatorsmile <gatorsmile@...> Date: 2018-01-13T16:34:39Z fix ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org