[GitHub] spark pull request #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage...

2018-01-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20204#discussion_r161679707 --- Diff: python/run-tests-with-coverage --- @@ -0,0 +1,69 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #20153: [SPARK-22392][SQL] data source v2 columnar batch reader

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20153 **[Test build #86160 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86160/testReport)** for PR 20153 at commit

[GitHub] spark pull request #17280: [SPARK-19939] [ML] Add support for association ru...

2018-01-15 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/17280#discussion_r161679593 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -319,9 +323,11 @@ object FPGrowthModel extends MLReadable[FPGrowthModel] {

[GitHub] spark issue #20153: [SPARK-22392][SQL] data source v2 columnar batch reader

2018-01-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20153 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage...

2018-01-15 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20204#discussion_r161677975 --- Diff: python/run-tests-with-coverage --- @@ -0,0 +1,69 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation

[GitHub] spark issue #20232: [SPARK-23042][ML] Use OneHotEncoderModel to encode label...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20232 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86154/ Test PASSed. ---

[GitHub] spark issue #20232: [SPARK-23042][ML] Use OneHotEncoderModel to encode label...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20232 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20232: [SPARK-23042][ML] Use OneHotEncoderModel to encode label...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20232 **[Test build #86154 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86154/testReport)** for PR 20232 at commit

[GitHub] spark pull request #20163: [SPARK-22966][PYTHON][SQL] Python UDFs with retur...

2018-01-15 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20163#discussion_r161677327 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvaluatePython.scala --- @@ -144,6 +145,7 @@ object EvaluatePython {

[GitHub] spark issue #20267: [SPARK-23068][BUILD][RELEASE] doc build error from jekyl...

2018-01-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20267 I see. Thanks for info! So, is it ready to go anyway @felixcheung? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20265: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2018-01-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20265 I'll update the PR tomorrow. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20275: [SPARK-23085][ML] API parity for mllib.linalg.Vectors.sp...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20275 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20275: [SPARK-23085][ML] API parity for mllib.linalg.Vectors.sp...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20275 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86158/ Test PASSed. ---

[GitHub] spark issue #20275: [SPARK-23085][ML] API parity for mllib.linalg.Vectors.sp...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20275 **[Test build #86158 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86158/testReport)** for PR 20275 at commit

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20168 Overall looks good to me. Just some minor comments regarding with code comments and naming. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20168 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86156/ Test PASSed. ---

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20168 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20168 **[Test build #86156 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86156/testReport)** for PR 20168 at commit

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20265#discussion_r161671821 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -0,0 +1,195 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20164 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20164 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86157/ Test PASSed. ---

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20164 **[Test build #86157 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86157/testReport)** for PR 20164 at commit

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20265#discussion_r161672411 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -0,0 +1,195 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20265#discussion_r161672316 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -0,0 +1,195 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20265#discussion_r161671868 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -0,0 +1,195 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20265#discussion_r161671835 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -0,0 +1,195 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20223: [SPARK-23020][core] Fix races in launcher code, t...

2018-01-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20223 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20223: [SPARK-23020][core] Fix races in launcher code, test.

2018-01-15 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/20223 merging to master/2.3. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema test for f...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20266 **[Test build #86159 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86159/testReport)** for PR 20266 at commit

[GitHub] spark issue #20216: [SPARK-23024][WEB-UI]Spark ui about the contents of the ...

2018-01-15 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/20216 LGTM now --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema tes...

2018-01-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20266#discussion_r161668457 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema tes...

2018-01-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20266#discussion_r161668286 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20216: [SPARK-23024][WEB-UI]Spark ui about the contents of the ...

2018-01-15 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/20216 I agree with your second suggestion, before I did not understand what you mean, now I passed the test I understand what you mean. 1.In order for collapsible tables to persist on

[GitHub] spark issue #20275: [SPARK-23085][ML] API parity for mllib.linalg.Vectors.sp...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20275 **[Test build #86158 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86158/testReport)** for PR 20275 at commit

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20168 Btw, I think this isn't only to add non-integer image formats. So the PR title may be changed too. Like "Add ImageSchema support for all OpenCV image types"? ---

[GitHub] spark pull request #20275: [SPARK-23085][ML] API parity for mllib.linalg.Vec...

2018-01-15 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/20275 [SPARK-23085][ML] API parity for mllib.linalg.Vectors.sparse ## What changes were proposed in this pull request? `ML.Vectors#sparse(size: Int, elements: Seq[(Int, Double)])` support

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161665015 --- Diff: python/pyspark/ml/image.py --- @@ -128,11 +183,17 @@ def toNDArray(self, image): height = image.height width =

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161665832 --- Diff: python/pyspark/ml/tests.py --- @@ -1843,6 +1844,28 @@ def tearDown(self): class ImageReaderTest(SparkSessionTestCase): +

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161666778 --- Diff: python/pyspark/ml/tests.py --- @@ -1843,6 +1844,28 @@ def tearDown(self): class ImageReaderTest(SparkSessionTestCase): +

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161664859 --- Diff: python/pyspark/ml/image.py --- @@ -55,25 +72,66 @@ def imageSchema(self): """ if self._imageSchema is None: -

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161661795 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -37,20 +37,67 @@ import org.apache.spark.sql.types._ @Since("2.3.0")

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161664852 --- Diff: python/pyspark/ml/image.py --- @@ -55,25 +72,66 @@ def imageSchema(self): """ if self._imageSchema is None: -

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161664806 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -37,20 +37,67 @@ import org.apache.spark.sql.types._ @Since("2.3.0")

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161664786 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -37,20 +37,67 @@ import org.apache.spark.sql.types._ @Since("2.3.0")

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161663005 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -37,20 +37,67 @@ import org.apache.spark.sql.types._ @Since("2.3.0")

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161664731 --- Diff: python/pyspark/ml/image.py --- @@ -55,25 +72,66 @@ def imageSchema(self): """ if self._imageSchema is None: -

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161665337 --- Diff: python/pyspark/ml/image.py --- @@ -55,25 +72,66 @@ def imageSchema(self): """ if self._imageSchema is None: -

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161664566 --- Diff: python/pyspark/ml/image.py --- @@ -55,25 +72,66 @@ def imageSchema(self): """ if self._imageSchema is None: -

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161662177 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -37,20 +37,67 @@ import org.apache.spark.sql.types._ @Since("2.3.0")

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161664060 --- Diff: mllib/src/test/scala/org/apache/spark/ml/image/ImageSchemaSuite.scala --- @@ -83,7 +83,8 @@ class ImageSchemaSuite extends SparkFunSuite with

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161663481 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -37,20 +37,67 @@ import org.apache.spark.sql.types._ @Since("2.3.0")

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161664597 --- Diff: python/pyspark/ml/image.py --- @@ -55,25 +72,66 @@ def imageSchema(self): """ if self._imageSchema is None: -

[GitHub] spark issue #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION should chan...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86152/ Test PASSed. ---

[GitHub] spark issue #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION should chan...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20249 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION should chan...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20249 **[Test build #86152 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86152/testReport)** for PR 20249 at commit

[GitHub] spark pull request #20150: [SPARK-22956][SS] Bug fix for 2 streams union fai...

2018-01-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20150 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20150: [SPARK-22956][SS] Bug fix for 2 streams union failover s...

2018-01-15 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/20150 Thanks for your review! Shixiong --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20150: [SPARK-22956][SS] Bug fix for 2 streams union failover s...

2018-01-15 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20150 Thanks! Merging to master and 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20168 **[Test build #86156 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86156/testReport)** for PR 20168 at commit

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20164 **[Test build #86157 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86157/testReport)** for PR 20164 at commit

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-15 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/20164 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20258: [SPARK-23060][Python] New feature - apply method to exte...

2018-01-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20258 Oh, I see! Yea, they look quite same. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20258: [SPARK-23060][Python] New feature - apply method to exte...

2018-01-15 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20258 Is this similar to `Dataset.transform()` in Java/Scala API? But we don't have similar APIs for RDDs. --- - To unsubscribe,

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20164 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86151/ Test FAILed. ---

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20164 **[Test build #86151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86151/testReport)** for PR 20164 at commit

[GitHub] spark issue #20232: [SPARK-23042][ML] Use OneHotEncoderModel to encode label...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20232 **[Test build #86155 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86155/testReport)** for PR 20232 at commit

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-15 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r161660926 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -663,6 +665,95 @@ class FsHistoryProviderSuite extends

[GitHub] spark pull request #20232: [SPARK-23042][ML] Use OneHotEncoderModel to encod...

2018-01-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20232#discussion_r161660735 --- Diff: R/pkg/tests/fulltests/test_mllib_classification.R --- @@ -382,10 +382,10 @@ test_that("spark.mlp", { trainidxs <- base::sample(nrow(data),

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-15 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20211#discussion_r161659513 --- Diff: python/pyspark/sql/group.py --- @@ -233,6 +233,27 @@ def apply(self, udf): | 2| 1.1094003924504583|

[GitHub] spark pull request #20171: [SPARK-22978] [PySpark] Register Vectorized UDFs ...

2018-01-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20171#discussion_r161659245 --- Diff: python/pyspark/sql/catalog.py --- @@ -256,27 +258,58 @@ def registerFunction(self, name, f, returnType=StringType()): >>>

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-15 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20211#discussion_r161659200 --- Diff: python/pyspark/sql/group.py --- @@ -233,6 +233,27 @@ def apply(self, udf): | 2| 1.1094003924504583|

[GitHub] spark pull request #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregati...

2018-01-15 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19872#discussion_r161658799 --- Diff: python/pyspark/sql/functions.py --- @@ -2214,6 +2216,37 @@ def pandas_udf(f=None, returnType=None, functionType=None): ..

[GitHub] spark issue #20056: [SPARK-22878] [CORE] Count totalDroppedEvents for LiveLi...

2018-01-15 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20056 I see that `LiveListenerBus.droppedEventsCounter` and `lastReportTimestamp` are unused, so it certainly makes sense to clean them up one way or the other -- but that might mean we should delete

[GitHub] spark pull request #20171: [SPARK-22978] [PySpark] Register Vectorized UDFs ...

2018-01-15 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20171#discussion_r161657719 --- Diff: python/pyspark/sql/catalog.py --- @@ -256,27 +258,58 @@ def registerFunction(self, name, f, returnType=StringType()): >>>

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20257 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20257 **[Test build #86153 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86153/testReport)** for PR 20257 at commit

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20257 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86153/ Test PASSed. ---

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-15 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/20168 @MrBago @tomasatdatabricks the changes look good to me, I went through everything one more time, I'll sign off as soon as the python tests are fixed (it looks like there were some style

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r161656633 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DecimalType.scala --- @@ -136,10 +137,52 @@ object DecimalType extends AbstractDataType

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161656541 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -37,20 +37,67 @@ import org.apache.spark.sql.types._

[GitHub] spark issue #20232: [SPARK-23042][ML] Use OneHotEncoderModel to encode label...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20232 **[Test build #86154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86154/testReport)** for PR 20232 at commit

[GitHub] spark pull request #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage...

2018-01-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20204#discussion_r161655584 --- Diff: python/run-tests-with-coverage --- @@ -0,0 +1,69 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-15 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r161656419 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -37,20 +37,67 @@ import org.apache.spark.sql.types._

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-15 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/20168 @MrBago @tomasatdatabricks I think the breaking changes are fine, the code was marked experimental and it is expected that the interfaces will change a lot initially based on early feedback.

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20257 **[Test build #86153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86153/testReport)** for PR 20257 at commit

[GitHub] spark issue #20272: [SPARK-23078] [CORE] allow Spark Thrift Server to run in...

2018-01-15 Thread ozzieba
Github user ozzieba commented on the issue: https://github.com/apache/spark/pull/20272 I'm getting stuck on https://github.com/apache-spark-on-k8s/spark-integration/blob/master/integration-test/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala#L106,

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20257 @MLnick Thanks for review. I think I've addressed all the comments. Please take a look for the updates. --- - To unsubscribe,

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r161655115 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DecimalPrecision.scala --- @@ -243,17 +279,43 @@ object DecimalPrecision

[GitHub] spark issue #20208: [SPARK-23007][SQL][TEST] Add schema evolution test suite...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86144/ Test PASSed. ---

[GitHub] spark issue #20208: [SPARK-23007][SQL][TEST] Add schema evolution test suite...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20208 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20208: [SPARK-23007][SQL][TEST] Add schema evolution test suite...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20208 **[Test build #86144 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86144/testReport)** for PR 20208 at commit

[GitHub] spark pull request #20171: [SPARK-22978] [PySpark] Register Vectorized UDFs ...

2018-01-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20171#discussion_r161654514 --- Diff: python/pyspark/sql/catalog.py --- @@ -256,27 +258,58 @@ def registerFunction(self, name, f, returnType=StringType()): >>>

[GitHub] spark issue #20265: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20265 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86143/ Test PASSed. ---

[GitHub] spark issue #20265: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2018-01-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20265 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20265: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2018-01-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20265 **[Test build #86143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86143/testReport)** for PR 20265 at commit

[GitHub] spark pull request #20171: [SPARK-22978] [PySpark] Register Vectorized UDFs ...

2018-01-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20171#discussion_r161654136 --- Diff: python/pyspark/sql/catalog.py --- @@ -256,27 +258,58 @@ def registerFunction(self, name, f, returnType=StringType()): >>>

[GitHub] spark pull request #20273: [SPARK-23000] Use fully qualified table names in ...

2018-01-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20273 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema test for f...

2018-01-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20266 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20266: [SPARK-23072][SQL][TEST] Add a Unicode schema tes...

2018-01-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20266#discussion_r161653628 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache

  1   2   3   4   >