[GitHub] spark pull request #20524: [SPARK-23345][SQL] Remove open stream record even...

2018-02-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20524 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20535: [SPARK-23341][SQL] define some standard options f...

2018-02-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/20535#discussion_r166701501 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/DataSourceOptions.java --- @@ -27,6 +27,39 @@ /** * An immutable string-to-string

[GitHub] spark issue #19145: [spark-21933][yarn] Spark Streaming request more executo...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19145 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87167 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87167/testReport)** for PR 20474 at commit

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87167/ Test FAILed. ---

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20531: [SPARK-23352][PYTHON] Explicitly specify supported types...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20531 **[Test build #87164 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87164/testReport)** for PR 20531 at commit

[GitHub] spark issue #20531: [SPARK-23352][PYTHON] Explicitly specify supported types...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20531 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87164/ Test PASSed. ---

[GitHub] spark issue #20531: [SPARK-23352][PYTHON] Explicitly specify supported types...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20531 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87165/ Test PASSed. ---

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20534 **[Test build #87165 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87165/testReport)** for PR 20534 at commit

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87172 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87172/testReport)** for PR 20474 at commit

[GitHub] spark pull request #20535: [SPARK-23341][SQL] define some standard options f...

2018-02-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20535#discussion_r166709990 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -171,7 +171,8 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20445 Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20536: Revert "[SPARK-22279][SQL] Turn on spark.sql.hive...

2018-02-07 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/20536 Revert "[SPARK-22279][SQL] Turn on spark.sql.hive.convertMetastoreOrc by default ## What changes were proposed in this pull request? This is to revert the changes made in

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20490 **[Test build #87169 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87169/testReport)** for PR 20490 at commit

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r166716818 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,44 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark pull request #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSo...

2018-02-07 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20445#discussion_r166725859 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala --- @@ -149,18 +149,12 @@ case class MemoryStream[A :

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20536 After a discussion with @marmbrus @rxin @yhuai , we decided to revert this for avoiding any regression and blocking the release of Spark 2.3. cc @sameeragarwal @dongjoon-hyun @cloud-fan

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20484 Just FYI, https://github.com/apache/spark/pull/20536 is reverting the conf `convertMetastoreOrc` back `false`. However, we still can turn it on by default in 2.3 after we fix the regression.

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20536 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20536 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/676/

[GitHub] spark pull request #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSo...

2018-02-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20445#discussion_r166736890 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala --- @@ -149,18 +149,12 @@ case class MemoryStream[A : Encoder](id:

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20445 **[Test build #87176 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87176/testReport)** for PR 20445 at commit

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/675/

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/677/

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/20499 I'd fix this in 2.3, and 2.2.1 as well. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-02-07 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20373 @HyukjinKwon would it be good to update this PR to match the upcoming 4.3 release you are working on? If the code is the same, then just updating the title/description so it is clear ---

[GitHub] spark issue #20519: [Spark-23240][python] Don't let python site customizatio...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20519 **[Test build #87173 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87173/testReport)** for PR 20519 at commit

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20445 **[Test build #87174 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87174/testReport)** for PR 20445 at commit

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20445 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87169/ Test PASSed. ---

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20474 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSo...

2018-02-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20445#discussion_r166720161 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/ForeachSinkSuite.scala --- @@ -46,49 +46,34 @@ class ForeachSinkSuite extends

[GitHub] spark issue #20530: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and red...

2018-02-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20530 @wujianping10043419 this change is too trivial to merge, I am sorry. The main reason for not merging is because this will mess up the git blame for the given code, which is sometimes very useful

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20445 LGTM pending passing run of that HiveDDLSuite test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20536 **[Test build #87175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87175/testReport)** for PR 20536 at commit

[GitHub] spark issue #20424: [Spark-23240][python] Better error message when extraneo...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20424 **[Test build #87179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87179/testReport)** for PR 20424 at commit

[GitHub] spark pull request #20533: [SPARK-23300][TESTS][BRANCH-2.3] Prints out if Pa...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/20533 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20534 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20534 **[Test build #87180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87180/testReport)** for PR 20534 at commit

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/680/

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-07 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20382 Sure, I will do it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-02-07 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20274 @liufengdb Can you verify this one? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166805463 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -228,14 +231,23 @@ private[spark] class EventLoggingListener(

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166805197 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -53,10 +53,21 @@ package object config { .booleanConf

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166805138 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -53,10 +53,21 @@ package object config { .booleanConf

[GitHub] spark issue #19894: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-07 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19894 I'm going to backport this to 2.2 since it's a correctness bug. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20538 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20538 **[Test build #87181 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87181/testReport)** for PR 20538 at commit

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20538 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87181/ Test PASSed. ---

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20484 I see. I removed `spark.sql.hive.convertMetastoreOrc` and Hive ORC table stuff from this PR accordingly. We can add that later if we fix the regresson of `convertMetastoreOrc/Parquet`. ---

[GitHub] spark issue #20524: [SPARK-23345][SQL] Remove open stream record even closin...

2018-02-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20524 LGTM Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20484 Oh, I thought in this way, @tgravescs . - For ORC files, the parameter names (ORC/Hive) are the same as we see https://github.com/apache/spark/pull/20517 . - For Hive tables, we don't

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166774153 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -190,9 +190,13 @@ object FileFormatWriter

[GitHub] spark pull request #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20472#discussion_r166771033 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -1001,11 +996,18 @@ private[spark] object RandomForest extends

[GitHub] spark pull request #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20472#discussion_r166771387 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -1001,11 +996,18 @@ private[spark] object RandomForest extends

[GitHub] spark pull request #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20472#discussion_r166770380 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -931,7 +925,8 @@ private[spark] object RandomForest extends Logging {

[GitHub] spark pull request #20537: [SPARK-23314][PYTHON] Add ambiguous=False when lo...

2018-02-07 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20537#discussion_r166810415 --- Diff: python/pyspark/sql/types.py --- @@ -1730,7 +1730,28 @@ def _check_series_convert_timestamps_internal(s, timezone): # TODO: handle

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20538 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/681/

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20538 **[Test build #87181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87181/testReport)** for PR 20538 at commit

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20538 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20533: [SPARK-23300][TESTS][BRANCH-2.3] Prints out if Pandas an...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20533 Merged to branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Pyt...

2018-02-07 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/20538 [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 without pandas. ## What changes were proposed in this pull request? This is a followup pr of #20487. When importing module

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20538 cc @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20534 @HyukjinKwon You can include the fix #20538 or backport it after this is merged. It's up to you. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2018-02-07 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r166815274 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -20,6 +20,7 @@ import javax.annotation.Nullable;

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20534 Yea, will deal with it. Thanks for the reminder! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166817939 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -53,10 +53,21 @@ package object config { .booleanConf

[GitHub] spark pull request #20518: [SPARK-22119][FOLLOWUP][ML] Use spherical KMeans ...

2018-02-07 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/20518#discussion_r166813909 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/KMeans.scala --- @@ -745,4 +763,27 @@ private[spark] class CosineDistanceMeasure

[GitHub] spark issue #20516: [SPARK-23343][CORE][TEST] Increase the exception test fo...

2018-02-07 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/20516 @srowen, if you don't set the spark.testing for true, the default value for spark.port.maxRetries is not 100, but 16, so in verifyServicePort function, actualPort should be <=

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2018-02-07 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r166815823 --- Diff: common/unsafe/src/test/java/org/apache/spark/unsafe/PlatformUtilSuite.java --- @@ -134,4 +135,24 @@ public void memoryDebugFillEnabledInTest() {

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19077 **[Test build #87183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87183/testReport)** for PR 19077 at commit

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20373 Yup, now the codes of branch "0.4.x" in cloudpickle is the same with the current PR. Was thinking of letting you know after 0.4.3. Please give me few days ... :-). ---

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-02-07 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20373 Sounds good! No rush, I'll keep an eye out for the release --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20537: [SPARK-23314][PYTHON] Add ambiguous=False when lo...

2018-02-07 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20537#discussion_r166811270 --- Diff: python/pyspark/sql/types.py --- @@ -1730,7 +1730,28 @@ def _check_series_convert_timestamps_internal(s, timezone): # TODO: handle

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-07 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166814031 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,53 @@ class

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20534 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2018-02-07 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r166817332 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/ByteArrayMethods.java --- @@ -40,6 +40,15 @@ public static int

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166818413 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -228,14 +231,23 @@ private[spark] class EventLoggingListener(

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20525 **[Test build #87182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87182/testReport)** for PR 20525 at commit

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/682/

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166817978 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -228,14 +231,23 @@ private[spark] class EventLoggingListener(

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19077 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/683/

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19077 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20536 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87175/ Test PASSed. ---

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166766796 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,53 @@ class

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166767449 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,53 @@ class

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166767860 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,53 @@ class

[GitHub] spark issue #20519: [Spark-23240][python] Don't let python site customizatio...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20519 **[Test build #87173 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87173/testReport)** for PR 20519 at commit

[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/679/

[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20537 cc @felixcheung @BryanCutler @ueshin @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20537 This is pretty important bug fix that we should try to get in Spark 2.3...Thanks @felixcheung for reporting this! --- - To

  1   2   3   4   5   6   >