[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-15 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r151170727 --- Diff: python/pyspark/sql/functions.py --- @@ -2247,16 +2142,20 @@ def pandas_udf(f=None, returnType=StringType()): | 8| JOHN

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-15 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r151166483 --- Diff: python/pyspark/sql/functions.py --- @@ -2050,7 +2050,6 @@ def map_values(col): # User Defined Function

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-14 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r151042155 --- Diff: python/pyspark/sql/functions.py --- @@ -2247,16 +2142,20 @@ def pandas_udf(f=None, returnType=StringType()): | 8| JOHN DOE|

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-14 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150963595 --- Diff: python/pyspark/sql/functions.py --- @@ -2279,7 +2172,38 @@ def pandas_udf(f=None, returnType=StringType()): .. note:: The user-de

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150700736 --- Diff: python/pyspark/sql/functions.py --- @@ -2049,132 +2051,13 @@ def map_values(col): # User Defined Functi

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-13 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150699602 --- Diff: python/pyspark/sql/functions.py --- @@ -2049,132 +2051,13 @@ def map_values(col): # User Defined Functio

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150696878 --- Diff: python/pyspark/sql/functions.py --- @@ -2049,132 +2051,13 @@ def map_values(col): # User Defined Functi

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150695345 --- Diff: python/pyspark/sql/functions.py --- @@ -2049,132 +2051,13 @@ def map_values(col): # User Defined Functi

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-13 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150662476 --- Diff: python/pyspark/sql/functions.py --- @@ -2049,132 +2051,13 @@ def map_values(col): # User Defined Functio

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150648631 --- Diff: python/pyspark/sql/functions.py --- @@ -2049,132 +2051,13 @@ def map_values(col): # User Defined Function

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150571488 --- Diff: python/pyspark/sql/functions.py --- @@ -2279,7 +2172,38 @@ def pandas_udf(f=None, returnType=StringType()): .. note:: The user-def

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150311258 --- Diff: python/pyspark/sql/functions.py --- @@ -2049,133 +2049,18 @@ def map_values(col): # User Defined Functio

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150311217 --- Diff: python/pyspark/sql/group.py --- @@ -260,19 +259,19 @@ def wrapped(*cols): import pandas as pd result = func(pd

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150311184 --- Diff: python/pyspark/sql/udf.py --- @@ -0,0 +1,136 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contribu

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r150311170 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -23,14 +23,15 @@ import scala.collection.JavaConverters._

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r149873461 --- Diff: python/pyspark/sql/udf.py --- @@ -0,0 +1,136 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r149873412 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -23,14 +23,15 @@ import scala.collection.JavaConverters._ i

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148957066 --- Diff: python/pyspark/sql/group.py --- @@ -260,19 +259,19 @@ def wrapped(*cols): import pandas as pd result = func(p

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148811820 --- Diff: python/pyspark/sql/group.py --- @@ -214,11 +214,11 @@ def apply(self, udf): :param udf: A function object returned by :meth:

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148810396 --- Diff: python/pyspark/sql/udf.py --- @@ -0,0 +1,136 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contribu

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148810140 --- Diff: python/pyspark/sql/udf.py --- @@ -0,0 +1,136 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contribut

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148807539 --- Diff: python/pyspark/sql/group.py --- @@ -214,11 +214,11 @@ def apply(self, udf): :param udf: A function object returned by :meth:`

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148807433 --- Diff: python/pyspark/sql/functions.py --- @@ -2208,16 +2093,26 @@ def udf(f=None, returnType=StringType()): | 8| JOHN DOE|

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148806731 --- Diff: python/pyspark/rdd.py --- @@ -56,6 +56,22 @@ __all__ = ["RDD"] +class PythonEvalType(object): +""" +Evaluati

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148805721 --- Diff: python/pyspark/sql/functions.py --- @@ -2279,7 +2174,36 @@ def pandas_udf(f=None, returnType=StringType()): .. note:: The user-de

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148805057 --- Diff: python/pyspark/sql/functions.py --- @@ -2049,133 +2049,18 @@ def map_values(col): # User Defined Function

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148804761 --- Diff: python/pyspark/sql/functions.py --- @@ -2279,7 +2174,36 @@ def pandas_udf(f=None, returnType=StringType()): .. note:: The user-def

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148803174 --- Diff: python/pyspark/sql/functions.py --- @@ -2208,16 +2093,26 @@ def udf(f=None, returnType=StringType()): | 8| JOHN DOE|

[GitHub] spark pull request #19630: wip: [SPARK-22409] Introduce function type argume...

2017-11-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r148800791 --- Diff: python/pyspark/rdd.py --- @@ -56,6 +56,22 @@ __all__ = ["RDD"] +class PythonEvalType(object): +""" +Evaluati