[GitHub] spark pull request #21034: [SPARK-23926][SQL] Extending reverse function to ...

2018-04-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21034#discussion_r180602624 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -212,6 +213,96 @@ case class

[GitHub] spark pull request #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and...

2018-04-10 Thread BryanCutler
Github user BryanCutler closed the pull request at: https://github.com/apache/spark/pull/21013 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-10 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21013 Closing this now, will reopen when the issue is resolved in the next Arrow release --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpa...

2018-04-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21007#discussion_r180602148 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TestQueryExecutionListener.scala --- @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpark as a...

2018-04-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21007 will address the comments soon. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Opt...

2018-04-10 Thread henryr
Github user henryr commented on a diff in the pull request: https://github.com/apache/spark/pull/20560#discussion_r179003707 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -274,3 +279,7 @@ abstract class BinaryNode

[GitHub] spark pull request #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Opt...

2018-04-10 Thread henryr
Github user henryr commented on a diff in the pull request: https://github.com/apache/spark/pull/20560#discussion_r180601594 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -733,6 +735,17 @@ object EliminateSorts extends

[GitHub] spark pull request #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Opt...

2018-04-10 Thread henryr
Github user henryr commented on a diff in the pull request: https://github.com/apache/spark/pull/20560#discussion_r180592716 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala --- @@ -22,10 +22,11 @@ import org.apache.spark.sql.{execution, Row}

[GitHub] spark issue #21026: [SPARK-23951][SQL] Use actual java class instead of stri...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21026 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2182/

[GitHub] spark issue #21026: [SPARK-23951][SQL] Use actual java class instead of stri...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21026 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21026: [SPARK-23951][SQL] Use actual java class instead of stri...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21026 **[Test build #89160 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89160/testReport)** for PR 21026 at commit

[GitHub] spark pull request #21026: [SPARK-23951][SQL] Use actual java class instead ...

2018-04-10 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/21026#discussion_r180598651 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/nullExpressions.scala --- @@ -352,12 +346,10 @@ case class

[GitHub] spark issue #21026: [SPARK-23951][SQL] Use actual java class instead of stri...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21026 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2181/

[GitHub] spark issue #21026: [SPARK-23951][SQL] Use actual java class instead of stri...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21026 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21026: [SPARK-23951][SQL] Use actual java class instead of stri...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21026 **[Test build #89159 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89159/testReport)** for PR 21026 at commit

[GitHub] spark pull request #21026: [SPARK-23951][SQL] Use actual java class instead ...

2018-04-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21026#discussion_r180596423 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/nullExpressions.scala --- @@ -352,12 +346,10 @@ case class

[GitHub] spark pull request #21026: [SPARK-23951][SQL] Use actual java class instead ...

2018-04-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21026#discussion_r180593331 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -0,0 +1,145 @@ +/* + * Licensed

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89154/ Test FAILed. ---

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #89154 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89154/testReport)** for PR 20959 at commit

[GitHub] spark issue #20280: [SPARK-22232][PYTHON][SQL] Fixed Row pickling to include...

2018-04-10 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20280 Hey @holdenk , yeah I've been meaning to circle back to this and get some kind of resolution. I'll try to take another look later this week. ---

[GitHub] spark issue #21003: [SPARK-23871][ML][PYTHON]add python api for VectorAssemb...

2018-04-10 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/21003 @jkbradley Thank you very much for your help! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21013: [WIP][SPARK-23874][SQL][PYTHON] Upgrade Arrow and pyarro...

2018-04-10 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21013 Yeah, we should be able to keep the minimum version the same, afaik there is no break in compatibility. @HyukjinKwon so do you suggest we should test with the minimum version and current

[GitHub] spark pull request #21003: [SPARK-23871][ML][PYTHON]add python api for Vecto...

2018-04-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21003 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89155/ Test FAILed. ---

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #89155 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89155/testReport)** for PR 20959 at commit

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21003: [SPARK-23871][ML][PYTHON]add python api for VectorAssemb...

2018-04-10 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/21003 Merging with master Thanks @huaxingao ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpa...

2018-04-10 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21007#discussion_r180584581 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TestQueryExecutionListener.scala --- @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpa...

2018-04-10 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21007#discussion_r180584207 --- Diff: python/pyspark/sql/tests.py --- @@ -3062,6 +3062,73 @@ def test_sparksession_with_stopped_sparkcontext(self): sc.stop()

[GitHub] spark pull request #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpa...

2018-04-10 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21007#discussion_r180583899 --- Diff: python/pyspark/sql/tests.py --- @@ -3062,6 +3062,73 @@ def test_sparksession_with_stopped_sparkcontext(self): sc.stop()

[GitHub] spark pull request #21007: [SPARK-23942][PYTHON][SQL] Makes collect in PySpa...

2018-04-10 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21007#discussion_r180585895 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TestQueryExecutionListener.scala --- @@ -0,0 +1,45 @@ +/* --- End diff -- Is

[GitHub] spark issue #20984: [SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20984 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20984: [SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20984 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89151/ Test PASSed. ---

[GitHub] spark issue #20984: [SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20984 **[Test build #89151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89151/testReport)** for PR 20984 at commit

[GitHub] spark issue #20984: [SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20984 **[Test build #89158 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89158/testReport)** for PR 20984 at commit

[GitHub] spark pull request #21026: [SPARK-23951][SQL] Use actual java class instead ...

2018-04-10 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/21026#discussion_r180575894 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateSafeProjection.scala --- @@ -107,7 +110,7 @@ object

[GitHub] spark pull request #21026: [SPARK-23951][SQL] Use actual java class instead ...

2018-04-10 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/21026#discussion_r180583517 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -0,0 +1,145 @@ +/* + * Licensed

[GitHub] spark pull request #21026: [SPARK-23951][SQL] Use actual java class instead ...

2018-04-10 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/21026#discussion_r180575717 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala --- @@ -292,8 +292,7 @@ object

[GitHub] spark pull request #21026: [SPARK-23951][SQL] Use actual java class instead ...

2018-04-10 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/21026#discussion_r180575964 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateSafeProjection.scala --- @@ -76,7 +78,7 @@ object

[GitHub] spark pull request #21026: [SPARK-23951][SQL] Use actual java class instead ...

2018-04-10 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/21026#discussion_r180575803 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateSafeProjection.scala --- @@ -128,7 +131,7 @@ object

[GitHub] spark issue #20984: [SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20984 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2180/

[GitHub] spark issue #20984: [SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20984 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20981: [SPARK-23873][SQL] Use accessors in interpreted L...

2018-04-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20981#discussion_r180583831 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/InternalRow.scala --- @@ -119,4 +119,28 @@ object InternalRow { case v: MapData

[GitHub] spark pull request #20981: [SPARK-23873][SQL] Use accessors in interpreted L...

2018-04-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20981#discussion_r180583775 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -65,11 +65,19 @@ trait

[GitHub] spark issue #20984: [SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData

2018-04-10 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20984 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20992: [SPARK-23779][SQL] TaskMemoryManager and UnsafeSorter re...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20992 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89147/ Test PASSed. ---

[GitHub] spark issue #20992: [SPARK-23779][SQL] TaskMemoryManager and UnsafeSorter re...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20992 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20992: [SPARK-23779][SQL] TaskMemoryManager and UnsafeSorter re...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20992 **[Test build #89147 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89147/testReport)** for PR 20992 at commit

[GitHub] spark issue #21003: [SPARK-23871][ML][PYTHON]add python api for VectorAssemb...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21003 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89156/ Test PASSed. ---

[GitHub] spark issue #21003: [SPARK-23871][ML][PYTHON]add python api for VectorAssemb...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21003 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21003: [SPARK-23871][ML][PYTHON]add python api for VectorAssemb...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21003 **[Test build #89156 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89156/testReport)** for PR 21003 at commit

[GitHub] spark issue #21026: [SPARK-23951][SQL] Use actual java class instead of stri...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21026 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21026: [SPARK-23951][SQL] Use actual java class instead of stri...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21026 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2179/

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20874 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20874 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89149/ Test FAILed. ---

[GitHub] spark issue #21026: [SPARK-23951][SQL] Use actual java class instead of stri...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21026 **[Test build #89157 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89157/testReport)** for PR 21026 at commit

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20874 **[Test build #89149 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89149/testReport)** for PR 20874 at commit

[GitHub] spark issue #20984: [SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20984 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89152/ Test FAILed. ---

[GitHub] spark issue #20984: [SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20984 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20984: [SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20984 **[Test build #89152 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89152/testReport)** for PR 20984 at commit

[GitHub] spark issue #21024: [SPARK-23917][SQL] Add array_max function

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21024 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21024: [SPARK-23917][SQL] Add array_max function

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21024 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89145/ Test PASSed. ---

[GitHub] spark issue #21024: [SPARK-23917][SQL] Add array_max function

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21024 **[Test build #89145 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89145/testReport)** for PR 21024 at commit

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the external c...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20795 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the external c...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20795 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89142/ Test PASSed. ---

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21034 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the external c...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20795 **[Test build #89142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89142/testReport)** for PR 20795 at commit

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21034 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21034: [SPARK-23926][SQL] Extending reverse function to ...

2018-04-10 Thread mn-mikke
GitHub user mn-mikke opened a pull request: https://github.com/apache/spark/pull/21034 [SPARK-23926][SQL] Extending reverse function to support ArrayType arguments ## What changes were proposed in this pull request? This PR extends reverse functions to be able to operate

[GitHub] spark issue #21003: [SPARK-23871][ML][PYTHON]add python api for VectorAssemb...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21003 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21003: [SPARK-23871][ML][PYTHON]add python api for VectorAssemb...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21003 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2178/

[GitHub] spark issue #21003: [SPARK-23871][ML][PYTHON]add python api for VectorAssemb...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21003 **[Test build #89156 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89156/testReport)** for PR 21003 at commit

[GitHub] spark issue #21003: [SPARK-23871][ML][PYTHON]add python api for VectorAssemb...

2018-04-10 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/21003 @jkbradley Thanks for your comment. I will add "and NaN" in the doc. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21031: [SPARK-23923][SQL] Add cardinality function

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21031 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21031: [SPARK-23923][SQL] Add cardinality function

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21031 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89150/ Test FAILed. ---

[GitHub] spark issue #21031: [SPARK-23923][SQL] Add cardinality function

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21031 **[Test build #89150 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89150/testReport)** for PR 21031 at commit

[GitHub] spark issue #19881: [SPARK-22683][CORE] Add a fullExecutorAllocationDivisor ...

2018-04-10 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/19881 I thought about this more, and I actually think something like this makes more sense: `executorAllocationRatio`. Basically it is just a ratio that determines how aggressive we want Spark to request

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #89155 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89155/testReport)** for PR 20959 at commit

[GitHub] spark issue #20871: [SPARK-23762][SQL] UTF8StringBuffer uses MemoryBlock

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20871 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20871: [SPARK-23762][SQL] UTF8StringBuffer uses MemoryBlock

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20871 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89148/ Test FAILed. ---

[GitHub] spark issue #21025: [SPARK-23918][SQL] Add array_min function

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21025 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20871: [SPARK-23762][SQL] UTF8StringBuffer uses MemoryBlock

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20871 **[Test build #89148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89148/testReport)** for PR 20871 at commit

[GitHub] spark issue #21025: [SPARK-23918][SQL] Add array_min function

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21025 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89140/ Test PASSed. ---

[GitHub] spark issue #21025: [SPARK-23918][SQL] Add array_min function

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21025 **[Test build #89140 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89140/testReport)** for PR 21025 at commit

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #89154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89154/testReport)** for PR 20959 at commit

[GitHub] spark pull request #21026: [SPARK-23951][SQL] Use actual java class instead ...

2018-04-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21026#discussion_r180554710 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -331,7 +333,7 @@ class

[GitHub] spark pull request #20983: [SPARK-23747][Structured Streaming] Add EpochCoor...

2018-04-10 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20983#discussion_r180553748 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/EpochCoordinatorSuite.scala --- @@ -0,0 +1,225 @@ +/* + * Licensed to the

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89144/ Test FAILed. ---

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21021 **[Test build #89144 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89144/testReport)** for PR 21021 at commit

[GitHub] spark issue #21003: [SPARK-23871][ML][PYTHON]add python api for VectorAssemb...

2018-04-10 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/21003 LGTM But would you mind making one fix in the doc here and in Scala? "Param for how to handle invalid data (NULL values)" should actually read "Param for how to handle invalid data (NULL

[GitHub] spark pull request #20989: [SPARK-23529][K8s] Support mounting hostPath volu...

2018-04-10 Thread madanadit
Github user madanadit closed the pull request at: https://github.com/apache/spark/pull/20989 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21033: [SPARK-19320][MESOS][WIP]allow specifying a hard limit o...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21033 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21033: [SPARK-19320][MESOS][WIP]allow specifying a hard limit o...

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21033 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21033: [SPARK-19320][MESOS][WIP]allow specifying a hard ...

2018-04-10 Thread yanji84
GitHub user yanji84 opened a pull request: https://github.com/apache/spark/pull/21033 [SPARK-19320][MESOS][WIP]allow specifying a hard limit on number of gpus required in each spark executor when running on mesos ## What changes were proposed in this pull request?

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-04-10 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r180535990 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesSpec.scala --- @@ -14,17 +14,18 @@ * See the License

[GitHub] spark issue #21030: typo rawPredicition changed to rawPrediction

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21030 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89153/ Test PASSed. ---

[GitHub] spark issue #21030: typo rawPredicition changed to rawPrediction

2018-04-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21030 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21030: typo rawPredicition changed to rawPrediction

2018-04-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21030 **[Test build #89153 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89153/testReport)** for PR 21030 at commit

<    1   2   3   4   5   6   7   8   >