[GitHub] spark issue #23237: [SPARK-26279][CORE] Remove unused method in Logging

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23237 **[Test build #99749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99749/testReport)** for PR 23237 at commit

[GitHub] spark pull request #23207: [SPARK-26193][SQL] Implement shuffle write metric...

2018-12-05 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/23207#discussion_r239308829 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -170,13 +172,23 @@ class SQLMetricsSuite extends

[GitHub] spark pull request #23207: [SPARK-26193][SQL] Implement shuffle write metric...

2018-12-05 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/23207#discussion_r239308706 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLShuffleMetricsReporter.scala --- @@ -95,3 +96,59 @@ private[spark] object

[GitHub] spark issue #23237: [SPARK-26279][CORE] Remove unused method in Logging

2018-12-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23237 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #23207: [SPARK-26193][SQL] Implement shuffle write metric...

2018-12-05 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/23207#discussion_r239308197 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLShuffleMetricsReporter.scala --- @@ -95,3 +96,59 @@ private[spark] object

[GitHub] spark pull request #23207: [SPARK-26193][SQL] Implement shuffle write metric...

2018-12-05 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/23207#discussion_r239308082 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -38,12 +38,18 @@ case class CollectLimitExec(limit: Int, child:

[GitHub] spark pull request #23207: [SPARK-26193][SQL] Implement shuffle write metric...

2018-12-05 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/23207#discussion_r239308007 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -38,12 +38,18 @@ case class CollectLimitExec(limit: Int, child:

[GitHub] spark issue #23207: [SPARK-26193][SQL] Implement shuffle write metrics in SQ...

2018-12-05 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/23207 @xuanyuanking can you separate the prs to rename read side metric and the write side change? --- - To unsubscribe, e-mail:

[GitHub] spark issue #23213: [SPARK-26262][SQL] Runs SQLQueryTestSuite on mixed confi...

2018-12-05 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/23213 In short, we need to run the tests on the three patterns below, right? - wholeStage=true, factoryMode=CODEGEN_ONLY (default behaviour in Spark) - wholeStage=false, factoryMode=CODEGEN_ONLY

[GitHub] spark pull request #23225: [MINOR][CORE]Don't need to create an empty spill ...

2018-12-05 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/23225#discussion_r239305307 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -167,64 +167,67 @@ private void writeSortedFile(boolean

[GitHub] spark issue #22952: [SPARK-20568][SS] Provide option to clean up completed f...

2018-12-05 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/22952 @gaborgsomogyi That's really huge... Could you share how you tested? Like which FS (local/HDFS/S3/etc), directory structure, count of files... That would help me understanding the impact

[GitHub] spark issue #23236: [SPARK-26275][PYTHON][ML] Increases timeout for Streamin...

2018-12-05 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/23236 > Compared to the error values from the test failures above, they match up until the 10th batch but then these continue until the 16th where it has a timeout I suspect that might because as

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22612 **[Test build #99748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99748/testReport)** for PR 22612 at commit

[GitHub] spark issue #22275: [SPARK-25274][PYTHON][SQL] In toPandas with Arrow send u...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22275 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22275: [SPARK-25274][PYTHON][SQL] In toPandas with Arrow send u...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22275 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99743/ Test PASSed. ---

[GitHub] spark issue #22275: [SPARK-25274][PYTHON][SQL] In toPandas with Arrow send u...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22275 **[Test build #99743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99743/testReport)** for PR 22275 at commit

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23108 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23108 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23239: [SPARK-26021][SQL][followup] only deal with NaN a...

2018-12-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23239#discussion_r239302780 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriter.java --- @@ -198,11 +198,45 @@ protected final void

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23108 **[Test build #99747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99747/testReport)** for PR 23108 at commit

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/23108 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22514: [SPARK-25271][SQL] Hive ctas commands should use ...

2018-12-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22514#discussion_r239300131 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala --- @@ -95,9 +77,116 @@ case class

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread kevinyu98
Github user kevinyu98 commented on the issue: https://github.com/apache/spark/pull/23108 Retest please, I didn't change Pyspark code, and tested `--modules=pyspark-sql` in local machine, it is fine. Thanks --- - To

[GitHub] spark pull request #23236: [SPARK-26275][PYTHON][ML] Increases timeout for S...

2018-12-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23236 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99741/ Test PASSed. ---

[GitHub] spark issue #23236: [SPARK-26275][PYTHON][ML] Increases timeout for Streamin...

2018-12-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23236 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23236: [SPARK-26275][PYTHON][ML] Increases timeout for Streamin...

2018-12-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23236 Thanks guys .. I will try to take a look for it as well (although it's going to take relatively a long while). Let me get this in for now anyway. ---

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23241 **[Test build #99741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99741/testReport)** for PR 23241 at commit

[GitHub] spark issue #23236: [SPARK-26275][PYTHON][ML] Increases timeout for Streamin...

2018-12-05 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/23236 I agree with @BryanCutler's analysis and it looks a bit weird at few things about this test. I also think it is fine to increase the timeout for now. ---

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99742/ Test PASSed. ---

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23241 **[Test build #99742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99742/testReport)** for PR 23241 at commit

[GitHub] spark issue #23236: [SPARK-26275][PYTHON][ML] Increases timeout for Streamin...

2018-12-05 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/23236 True, the test is not that long under light resources. Locally, I saw a couple seconds difference with the changes I mentioned. The weird thing is the unmodified test completes after the 11th

[GitHub] spark issue #23225: [MINOR][CORE]Don't need to create an empty spill file wh...

2018-12-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23225 If a test passes without/with this changes, it wouldn't verify the regression. --- - To unsubscribe, e-mail:

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23108 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99746/ Test FAILed. ---

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23108 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23108 **[Test build #99746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99746/testReport)** for PR 23108 at commit

[GitHub] spark issue #23236: [SPARK-26275][PYTHON][ML] Increases timeout for Streamin...

2018-12-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23236 Thanks for a close look, @BryanCutler. I just increased the timeout because I was just trying to keep the test as is and fix it. The actual test doesn't look taking so much time but it looks it

[GitHub] spark issue #22759: [MINOR][SQL][DOC] Correct parquet nullability documentat...

2018-12-05 Thread dima-asana
Github user dima-asana commented on the issue: https://github.com/apache/spark/pull/22759 > @dima-asana can you rebase, and add a simple test case or else find one that does demonstrate the behavior here? done, sorry for the delay ---

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99740/ Test PASSed. ---

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23241 **[Test build #99740 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99740/testReport)** for PR 23241 at commit

[GitHub] spark pull request #22904: [SPARK-25887][K8S] Configurable K8S context suppo...

2018-12-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22904#discussion_r239272037 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala --- @@ -67,8 +66,16 @@

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22904 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22904 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22904 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5785/ ---

[GitHub] spark pull request #23159: [SPARK-26191][SQL] Control truncation of Spark pl...

2018-12-05 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/23159#discussion_r239269414 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1777,7 +1777,7 @@ class Analyzer(

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23241 Please use the PR title and summary to describe the solution, not the problem. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #23201: [SPARK-26246][SQL] Infer date and timestamp types...

2018-12-05 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/23201#discussion_r239269170 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala --- @@ -121,7 +121,18 @@ private[sql] class

[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21465 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99744/ Test PASSed. ---

[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21465 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21465 **[Test build #99744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99744/testReport)** for PR 21465 at commit

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23108 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22904 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5785/ ---

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23108 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23108 **[Test build #99746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99746/testReport)** for PR 23108 at commit

[GitHub] spark pull request #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE...

2018-12-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/23108#discussion_r239266908 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -186,6 +186,82 @@ abstract class OrcSuite

[GitHub] spark pull request #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE...

2018-12-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/23108#discussion_r239266720 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -2370,4 +2370,51 @@ class HiveDDLSuite ))

[GitHub] spark pull request #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE...

2018-12-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/23108#discussion_r239266747 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcSourceSuite.scala --- @@ -190,4 +190,12 @@ class HiveOrcSourceSuite extends

[GitHub] spark pull request #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE...

2018-12-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/23108#discussion_r239266791 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -2370,4 +2370,51 @@ class HiveDDLSuite ))

[GitHub] spark pull request #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE...

2018-12-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/23108#discussion_r239266816 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -2370,4 +2370,51 @@ class HiveDDLSuite ))

[GitHub] spark pull request #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE...

2018-12-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/23108#discussion_r239266775 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -2370,4 +2370,51 @@ class HiveDDLSuite ))

[GitHub] spark pull request #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE...

2018-12-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/23108#discussion_r239266673 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -186,6 +186,54 @@ abstract class OrcSuite

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22904 **[Test build #99745 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99745/testReport)** for PR 22904 at commit

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22904 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22904 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99745/ Test FAILed. ---

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22904 **[Test build #99745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99745/testReport)** for PR 22904 at commit

[GitHub] spark pull request #22514: [SPARK-25271][SQL] Hive ctas commands should use ...

2018-12-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22514#discussion_r239264673 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -181,62 +180,39 @@ case class RelationConversions(

[GitHub] spark pull request #22514: [SPARK-25271][SQL] Hive ctas commands should use ...

2018-12-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22514#discussion_r239264207 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala --- @@ -95,9 +77,116 @@ case class

[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21465 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-12-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22904 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21465 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-12-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22904 add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21465 **[Test build #99744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99744/testReport)** for PR 21465 at commit

[GitHub] spark issue #22905: [SPARK-25894][SQL] Add a ColumnarFileFormat type which r...

2018-12-05 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/22905 I think I've made my case for this patch as best I can. It does not appear this PR has unanimous support, but I continue to believe we should merge it to master. So where do we take it from here?

[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-05 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/21465 @BryanCutler Thank you very much for your review! I will submit changes soon. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22880: [SPARK-25407][SQL] Ensure we pass a compatible pruned sc...

2018-12-05 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/22880 Hi @dbtsai @HyukjinKwon @gatorsmile @viirya. Can we merge this to master? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239259444 --- Diff: R/pkg/R/mllib_clustering.R --- @@ -610,3 +616,58 @@ setMethod("write.ml", signature(object = "LDAModel", path = "character"),

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22612 **[Test build #4454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4454/testReport)** for PR 22612 at commit

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239258564 --- Diff: docs/ml-clustering.md --- @@ -265,3 +265,44 @@ Refer to the [R API docs](api/R/spark.gaussianMixture.html) for more details.

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23240 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99739/ Test PASSed. ---

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23240 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239258366 --- Diff: docs/ml-clustering.md --- @@ -265,3 +265,44 @@ Refer to the [R API docs](api/R/spark.gaussianMixture.html) for more details.

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239257840 --- Diff: docs/ml-clustering.md --- @@ -265,3 +265,44 @@ Refer to the [R API docs](api/R/spark.gaussianMixture.html) for more details.

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23240 **[Test build #99739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99739/testReport)** for PR 23240 at commit

[GitHub] spark issue #22275: [SPARK-25274][PYTHON][SQL] In toPandas with Arrow send u...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22275 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22275: [SPARK-25274][PYTHON][SQL] In toPandas with Arrow send u...

2018-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22275 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #23239: [SPARK-26021][SQL][followup] only deal with NaN a...

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23239#discussion_r239256853 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java --- @@ -174,11 +174,6 @@ public static float getFloat(Object object, long

[GitHub] spark pull request #23233: [SPARK-26233][SQL][BACKPORT-2.3] CheckOverflow wh...

2018-12-05 Thread mgaido91
Github user mgaido91 closed the pull request at: https://github.com/apache/spark/pull/23233 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #23234: [SPARK-26233][SQL][BACKPORT-2.2] CheckOverflow wh...

2018-12-05 Thread mgaido91
Github user mgaido91 closed the pull request at: https://github.com/apache/spark/pull/23234 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #23232: [SPARK-26233][SQL][BACKPORT-2.4] CheckOverflow wh...

2018-12-05 Thread mgaido91
Github user mgaido91 closed the pull request at: https://github.com/apache/spark/pull/23232 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23232: [SPARK-26233][SQL][BACKPORT-2.4] CheckOverflow when enco...

2018-12-05 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/23232 Done, thanks @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23156: [SPARK-24063][SS] Add maximum epoch queue threshold for ...

2018-12-05 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/23156 Rather than controlling the queue sizes it would be better to limit the max epoch backlog and fail the query once that threshold is reached. There already seems to be patch that attempted to

[GitHub] spark issue #22275: [SPARK-25274][PYTHON][SQL] In toPandas with Arrow send u...

2018-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22275 **[Test build #99743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99743/testReport)** for PR 22275 at commit

[GitHub] spark pull request #23141: [SPARK-26021][SQL][followup] add test for special...

2018-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23141#discussion_r239255684 --- Diff: common/unsafe/src/test/java/org/apache/spark/unsafe/PlatformUtilSuite.java --- @@ -165,10 +165,14 @@ public void

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239250376 --- Diff: docs/ml-clustering.md --- @@ -265,3 +265,44 @@ Refer to the [R API docs](api/R/spark.gaussianMixture.html) for more details.

[GitHub] spark issue #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/23072 @dongjoon-hyun Thank you very much for your review. I will make the changes soon. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-05 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/23072#discussion_r239250335 --- Diff: R/pkg/R/mllib_clustering.R --- @@ -610,3 +616,58 @@ setMethod("write.ml", signature(object = "LDAModel", path = "character"),

[GitHub] spark pull request #21465: [SPARK-24333][ML][PYTHON]Add fit with validation ...

2018-12-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21465#discussion_r239245388 --- Diff: python/pyspark/ml/regression.py --- @@ -705,12 +710,59 @@ def getNumTrees(self): return self.getOrDefault(self.numTrees)

<    1   2   3   4   5   6   7   >