[GitHub] spark pull request #20183: [SPARK-22986][Core] Fix/cache broadcast values

2018-01-10 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20183#discussion_r160881323 --- Diff: core/src/test/scala/org/apache/spark/broadcast/BroadcastSuite.scala --- @@ -153,6 +153,40 @@ class BroadcastSuite extends SparkFunSuite with

[GitHub] spark issue #20183: [SPARK-22986][Core] Fix/cache broadcast values

2018-01-10 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20183 Please also update the PR title and description. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20230: [SPARK-23038][TEST] Update docker/spark-test (JDK/OS)

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20230 **[Test build #85953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85953/testReport)** for PR 20230 at commit

[GitHub] spark pull request #20230: [SPARK-23038][TEST] Update docker/spark-test (JDK...

2018-01-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20230#discussion_r160880463 --- Diff: external/docker/spark-test/base/Dockerfile --- @@ -15,14 +15,14 @@ # limitations under the License. # -FROM

[GitHub] spark pull request #20153: [SPARK-22392][SQL] data source v2 columnar batch ...

2018-01-10 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20153#discussion_r160880016 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2ScanExec.scala --- @@ -37,40 +35,58 @@ import

[GitHub] spark pull request #20153: [SPARK-22392][SQL] data source v2 columnar batch ...

2018-01-10 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20153#discussion_r160877601 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/SupportsScanColumnarBatch.java --- @@ -0,0 +1,51 @@ +/* + * Licensed

[GitHub] spark pull request #20230: [SPARK-23038][TEST] Update docker/spark-test (JDK...

2018-01-10 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/20230 [SPARK-23038][TEST] Update docker/spark-test (JDK/OS) ## What changes were proposed in this pull request? This PR aims to update the followings in `docker/spark-test`. -

[GitHub] spark pull request #20199: [Spark-22967][TESTS]Fix VersionSuite's unit tests...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20199#discussion_r160879315 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala --- @@ -842,6 +842,7 @@ class VersionsSuite extends

[GitHub] spark pull request #20229: [SPARK-23037][ML] Update RFormula to use VectorSi...

2018-01-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20229#discussion_r160878813 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala --- @@ -199,6 +199,7 @@ class RFormula @Since("1.5.0") (@Since("1.5.0") override

[GitHub] spark pull request #20229: [SPARK-23037][ML] Update RFormula to use VectorSi...

2018-01-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20229#discussion_r160878716 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala --- @@ -228,22 +229,33 @@ class RFormula @Since("1.5.0") (@Since("1.5.0")

[GitHub] spark issue #20229: [SPARK-23037][ML] Update RFormula to use VectorSizeHint ...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20229 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85950/ Test PASSed. ---

[GitHub] spark issue #20229: [SPARK-23037][ML] Update RFormula to use VectorSizeHint ...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20229 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20229: [SPARK-23037][ML] Update RFormula to use VectorSizeHint ...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20229 **[Test build #85950 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85950/testReport)** for PR 20229 at commit

[GitHub] spark pull request #20229: [SPARK-23037][ML] Update RFormula to use VectorSi...

2018-01-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20229#discussion_r160877856 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala --- @@ -228,22 +229,33 @@ class RFormula @Since("1.5.0") (@Since("1.5.0")

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85947/ Test PASSed. ---

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #85947 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85947/testReport)** for PR 19872 at commit

[GitHub] spark issue #20229: [SPARK-23037][ML] Update RFormula to use VectorSizeHint ...

2018-01-10 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20229 Actually I think this can be separated in two different PRs. One for OneHotEncoderEstimator and one for VectorSizeHint. --- - To

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160877051 --- Diff: python/pyspark/sql/context.py --- @@ -578,6 +606,9 @@ def __init__(self, sqlContext): def register(self, name, f,

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20222 **[Test build #85952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85952/testReport)** for PR 20222 at commit

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20072 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85949/ Test FAILed. ---

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20072 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20072 **[Test build #85949 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85949/testReport)** for PR 20072 at commit

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160876638 --- Diff: python/pyspark/sql/context.py --- @@ -203,18 +203,46 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20222 Thanks for your review! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20229: [SPARK-23037][ML] Update RFormula to use VectorSizeHint ...

2018-01-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20229 we need to get it to kick off R tests - could you touch one of the files under R/? also please update PR to include [SPARKR] ---

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160873545 --- Diff: python/pyspark/sql/context.py --- @@ -203,18 +203,46 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160872774 --- Diff: python/pyspark/sql/catalog.py --- @@ -255,26 +255,67 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160873927 --- Diff: python/pyspark/sql/context.py --- @@ -578,6 +606,9 @@ def __init__(self, sqlContext): def register(self, name, f,

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160874070 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/executor/Dockerfile --- @@ -1,35 +0,0 @@ -# -# Licensed to the Apache

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160874300 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh --- @@ -0,0 +1,97 @@ +#!/bin/bash +# +# Licensed to

[GitHub] spark issue #20195: [SPARK-22972][SQL] Couldn't find corresponding Hive SerD...

2018-01-10 Thread xubo245
Github user xubo245 commented on the issue: https://github.com/apache/spark/pull/20195 ok @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20195: [SPARK-22972][SQL] Couldn't find corresponding Hi...

2018-01-10 Thread xubo245
Github user xubo245 closed the pull request at: https://github.com/apache/spark/pull/20195 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20229: Update RFormula to use VectorSizeHint & OneHotEncoderEst...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20229 **[Test build #85950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85950/testReport)** for PR 20229 at commit

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20214 **[Test build #85951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85951/testReport)** for PR 20214 at commit

[GitHub] spark issue #20206: [SPARK-19256][SQL] Remove ordering enforcement from `Fil...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20206 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85946/ Test PASSed. ---

[GitHub] spark issue #20206: [SPARK-19256][SQL] Remove ordering enforcement from `Fil...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20206 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20206: [SPARK-19256][SQL] Remove ordering enforcement from `Fil...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20206 **[Test build #85946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85946/testReport)** for PR 20206 at commit

[GitHub] spark pull request #20229: Update RFormula to use VectorSizeHint & OneHotEnc...

2018-01-10 Thread MrBago
GitHub user MrBago opened a pull request: https://github.com/apache/spark/pull/20229 Update RFormula to use VectorSizeHint & OneHotEncoderEstimator. ## What changes were proposed in this pull request? RFormula should use VectorSizeHint & OneHotEncoderEstimator in its

[GitHub] spark issue #20087: [SPARK-21786][SQL] The 'spark.sql.parquet.compression.co...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20087 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85944/ Test PASSed. ---

[GitHub] spark issue #20087: [SPARK-21786][SQL] The 'spark.sql.parquet.compression.co...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20087 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20087: [SPARK-21786][SQL] The 'spark.sql.parquet.compression.co...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20087 **[Test build #85944 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85944/testReport)** for PR 20087 at commit

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20214 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20214 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85948/ Test FAILed. ---

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20214 **[Test build #85948 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85948/testReport)** for PR 20214 at commit

[GitHub] spark pull request #20194: [SPARK-22999][SQL]'show databases like command' c...

2018-01-10 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/20194#discussion_r160869162 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -141,7 +141,7 @@ statement (LIKE?

[GitHub] spark issue #20195: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20195 @xubo245. Since it's merged, could you close your PR now? For the PR against old branches like this, we need to close the PR manually. ---

[GitHub] spark issue #20100: [SPARK-22913][SQL] Improved Hive Partition Pruning

2018-01-10 Thread ameent
Github user ameent commented on the issue: https://github.com/apache/spark/pull/20100 @srowen can you help find someone to review this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20100: [SPARK-22913][SQL] Improved Hive Partition Pruning

2018-01-10 Thread ameent
Github user ameent commented on the issue: https://github.com/apache/spark/pull/20100 Any updates on this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20194: [SPARK-22999][SQL]'show databases like command' c...

2018-01-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20194#discussion_r160867737 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -141,7 +141,7 @@ statement (LIKE?

[GitHub] spark pull request #20227: [SPARK-23035] Fix warning: TEMPORARY TABLE ... US...

2018-01-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20227#discussion_r160867160 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -814,7 +814,7 @@ abstract class DDLSuite extends

[GitHub] spark pull request #20227: [SPARK-23035] Fix warning: TEMPORARY TABLE ... US...

2018-01-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20227#discussion_r160867069 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/AlreadyExistException.scala --- @@ -33,6 +33,9 @@ class

[GitHub] spark issue #20228: [SPARK-23036] Add withGlobalTempView for testing and cor...

2018-01-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20228 BTW, please put '[SQL]' into your title. Then, your PR will be listed under SQL category here. - https://spark-prs.appspot.com/open-prs ---

[GitHub] spark issue #20226: [SPARK-23034][SQL][UI] Display tablename for `HiveTableS...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85945/ Test PASSed. ---

[GitHub] spark issue #20226: [SPARK-23034][SQL][UI] Display tablename for `HiveTableS...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20226: [SPARK-23034][SQL][UI] Display tablename for `HiveTableS...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #85945 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85945/testReport)** for PR 20226 at commit

[GitHub] spark pull request #20228: [SPARK-23036] Add withGlobalTempView for testing ...

2018-01-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20228#discussion_r160866670 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/GlobalTempViewSuite.scala --- @@ -140,8 +140,8 @@ class GlobalTempViewSuite extends

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160864450 --- Diff: python/pyspark/sql/context.py --- @@ -203,18 +203,46 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20211#discussion_r160864143 --- Diff: python/pyspark/sql/group.py --- @@ -233,6 +233,27 @@ def apply(self, udf): | 2| 1.1094003924504583|

[GitHub] spark pull request #20151: [SPARK-22959][PYTHON] Configuration to select the...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20151#discussion_r160864049 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -34,17 +34,39 @@ private[spark] class

[GitHub] spark issue #20072: [SPARK-22790][SQL] add a configurable factor to describe...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20072 **[Test build #85949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85949/testReport)** for PR 20072 at commit

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160863048 --- Diff: python/pyspark/sql/tests.py --- @@ -4085,33 +4091,50 @@ def test_vectorized_udf_timestamps_respect_session_timezone(self):

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160863023 --- Diff: python/pyspark/sql/tests.py --- @@ -4085,33 +4091,50 @@ def test_vectorized_udf_timestamps_respect_session_timezone(self):

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160862588 --- Diff: python/pyspark/sql/catalog.py --- @@ -255,26 +255,67 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20225 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20225 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85939/ Test FAILed. ---

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20211#discussion_r160862182 --- Diff: python/pyspark/sql/tests.py --- @@ -3995,23 +3995,49 @@ def test_coerce(self): self.assertFramesEqual(expected, result)

[GitHub] spark issue #20225: [SPARK-23033] Don't use task level retry for continuous ...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20225 **[Test build #85939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85939/testReport)** for PR 20225 at commit

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160862123 --- Diff: python/pyspark/sql/context.py --- @@ -203,18 +203,46 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-10 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20204 > @icexelloss, for #20204 (comment), yes, that's the way I usually use too. My worry is though I wonder if this is a proper official way to do it because I have been thinking this way is rather

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160861750 --- Diff: python/pyspark/sql/catalog.py --- @@ -255,26 +255,67 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark issue #20226: [SPARK-23034][SQL][UI] Display tablename for `HiveTableS...

2018-01-10 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/20226 @dongjoon-hyun : I have updated the PR description --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160861434 --- Diff: python/pyspark/sql/context.py --- @@ -203,18 +203,46 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark pull request #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20204#discussion_r160860487 --- Diff: python/test_coverage/sitecustomize.py --- @@ -0,0 +1,19 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20211#discussion_r160860320 --- Diff: python/pyspark/sql/group.py --- @@ -233,6 +233,27 @@ def apply(self, udf): | 2| 1.1094003924504583|

[GitHub] spark pull request #19885: [SPARK-22587] Spark job fails if fs.defaultFS and...

2018-01-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19885 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160859938 --- Diff: python/pyspark/sql/tests.py --- @@ -4147,6 +4170,21 @@ def test_simple(self): expected =

[GitHub] spark issue #19885: [SPARK-22587] Spark job fails if fs.defaultFS and applic...

2018-01-10 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19885 Let me merge to master and branch 2.3. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160858810 --- Diff: python/pyspark/sql/catalog.py --- @@ -255,26 +255,67 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160858570 --- Diff: python/pyspark/sql/catalog.py --- @@ -255,26 +255,67 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160858423 --- Diff: python/pyspark/sql/catalog.py --- @@ -255,26 +255,67 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark issue #20224: [SPARK-23032][SQL] Add a per-query codegenStageId to Who...

2018-01-10 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20224 We always need to turn on this? It seems this is debug info for developers? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20222 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85935/ Test FAILed. ---

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20222 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160858034 --- Diff: python/pyspark/sql/catalog.py --- @@ -255,26 +255,67 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark issue #20222: [SPARK-23028] Bump master branch version to 2.4.0-SNAPSH...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20222 **[Test build #85935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85935/testReport)** for PR 20222 at commit

[GitHub] spark issue #20228: [SPARK-23036] Add withGlobalTempView for testing and cor...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20228 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160857927 --- Diff: python/pyspark/sql/catalog.py --- @@ -255,26 +255,67 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark pull request #20217: [SPARK-23026] [PySpark] Add RegisterUDF to PySpar...

2018-01-10 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20217#discussion_r160857817 --- Diff: python/pyspark/sql/catalog.py --- @@ -255,26 +255,67 @@ def registerFunction(self, name, f, returnType=StringType()): >>> _ =

[GitHub] spark pull request #20228: [SPARK-23036] Add withGlobalTempView for testing ...

2018-01-10 Thread xubo245
GitHub user xubo245 opened a pull request: https://github.com/apache/spark/pull/20228 [SPARK-23036] Add withGlobalTempView for testing and correct some roper with view related method usage ## What changes were proposed in this pull request? Add

[GitHub] spark issue #20226: [SPARK-23034][SQL][UI] Display tablename for `HiveTableS...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20226: [SPARK-23034][SQL][UI] Display tablename for `HiveTableS...

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85943/ Test PASSed. ---

[GitHub] spark issue #20226: [SPARK-23034][SQL][UI] Display tablename for `HiveTableS...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #85943 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85943/testReport)** for PR 20226 at commit

[GitHub] spark issue #20223: [SPARK-23020][core] Fix races in launcher code, test.

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20223 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20223: [SPARK-23020][core] Fix races in launcher code, test.

2018-01-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20223 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85937/ Test PASSed. ---

[GitHub] spark issue #20223: [SPARK-23020][core] Fix races in launcher code, test.

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20223 **[Test build #85937 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85937/testReport)** for PR 20223 at commit

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20214 **[Test build #85948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85948/testReport)** for PR 20214 at commit

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-10 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20214 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20214: [SPARK-23023][SQL] Cast field data to strings in showStr...

2018-01-10 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20214 `org.apache.spark.sql.streaming.StreamingOuterJoinSuite` is flaky? (It seems this pr is not related to the test). --- - To

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #85947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85947/testReport)** for PR 19872 at commit

  1   2   3   4   5   >