[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20477 **[Test build #87189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87189/testReport)** for PR 20477 at commit

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20509 **[Test build #87188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87188/testReport)** for PR 20509 at commit

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20499 Will update this tonight --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166830970 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -190,9 +190,13 @@ object FileFormatWriter

[GitHub] spark issue #20539: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-07 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/20539 ping @jkbradley --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20539: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20539 **[Test build #87184 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87184/testReport)** for PR 20539 at commit

[GitHub] spark issue #20539: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20539 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20539: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20539 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87184/ Test PASSed. ---

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20535 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20535 **[Test build #87187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87187/testReport)** for PR 20535 at commit

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20535 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87187/ Test FAILed. ---

[GitHub] spark pull request #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive....

2018-02-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20536 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20536 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166829369 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -190,9 +190,13 @@ object FileFormatWriter

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2018-02-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r166829018 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/HeapMemoryAllocator.java --- @@ -46,9 +47,10 @@ private boolean shouldPool(long

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20535 **[Test build #87187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87187/testReport)** for PR 20535 at commit

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20535 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/687/

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20535 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20535 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87186/ Test FAILed. ---

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20535 **[Test build #87186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87186/testReport)** for PR 20535 at commit

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20535 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r166827975 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,130 @@ package

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r166827720 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,130 @@ package

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r166827284 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,130 @@ package

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20535 **[Test build #87186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87186/testReport)** for PR 20535 at commit

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/686/

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20535 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20535 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/685/

[GitHub] spark pull request #20537: [SPARK-23314][PYTHON] Add ambiguous=False when lo...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20537#discussion_r166826644 --- Diff: python/pyspark/sql/types.py --- @@ -1730,7 +1730,28 @@ def _check_series_convert_timestamps_internal(s, timezone): # TODO: handle

[GitHub] spark pull request #20537: [SPARK-23314][PYTHON] Add ambiguous=False when lo...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20537#discussion_r166826468 --- Diff: python/pyspark/sql/tests.py --- @@ -3638,6 +3638,21 @@ def test_createDataFrame_with_int_col_names(self):

[GitHub] spark pull request #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Pyt...

2018-02-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20538 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20534 **[Test build #87185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87185/testReport)** for PR 20534 at commit

[GitHub] spark pull request #20535: [SPARK-23341][SQL] define some standard options f...

2018-02-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20535#discussion_r166826237 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -171,7 +171,8 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20538 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20535: [SPARK-23341][SQL] define some standard options f...

2018-02-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20535#discussion_r166826138 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/DataSourceOptions.java --- @@ -27,6 +27,39 @@ /** * An immutable

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20534 Let me just pick up the followup here. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87180/ Test FAILed. ---

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20534 **[Test build #87180 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87180/testReport)** for PR 20534 at commit

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20535 > This should move the standard options to DataSourceV2Relation to avoid needing to instantiate DataSourceOptions wherever the relation is created. @rdblue We don't have this problem now,

[GitHub] spark issue #20424: [Spark-23240][python] Better error message when extraneo...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20424 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20424: [Spark-23240][python] Better error message when extraneo...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20424 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87179/ Test PASSed. ---

[GitHub] spark issue #20539: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20539 **[Test build #87184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87184/testReport)** for PR 20539 at commit

[GitHub] spark issue #20424: [Spark-23240][python] Better error message when extraneo...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20424 **[Test build #87179 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87179/testReport)** for PR 20424 at commit

[GitHub] spark issue #20539: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20539 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20539: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20539 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/684/

[GitHub] spark pull request #20539: [SPARK-22700][ML] Bucketizer.transform incorrectl...

2018-02-07 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/20539 [SPARK-22700][ML] Bucketizer.transform incorrectly drops row containing NaN - for branch-2.2 ## What changes were proposed in this pull request? for branch-2.2 only drops the rows

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166818413 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -228,14 +231,23 @@ private[spark] class EventLoggingListener(

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19077 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19077 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/683/

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166817978 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -228,14 +231,23 @@ private[spark] class EventLoggingListener(

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166817939 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -53,10 +53,21 @@ package object config { .booleanConf

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19077 **[Test build #87183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87183/testReport)** for PR 19077 at commit

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2018-02-07 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r166817332 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/ByteArrayMethods.java --- @@ -40,6 +40,15 @@ public static int

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20525 **[Test build #87182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87182/testReport)** for PR 20525 at commit

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/682/

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20538 **[Test build #87181 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87181/testReport)** for PR 20538 at commit

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20538 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87181/ Test PASSed. ---

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20538 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20534 Yea, will deal with it. Thanks for the reminder! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2018-02-07 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r166815823 --- Diff: common/unsafe/src/test/java/org/apache/spark/unsafe/PlatformUtilSuite.java --- @@ -134,4 +135,24 @@ public void memoryDebugFillEnabledInTest() {

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2018-02-07 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r166815274 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -20,6 +20,7 @@ import javax.annotation.Nullable;

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20534 @HyukjinKwon You can include the fix #20538 or backport it after this is merged. It's up to you. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20534 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-07 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166814031 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,53 @@ class

[GitHub] spark pull request #20518: [SPARK-22119][FOLLOWUP][ML] Use spherical KMeans ...

2018-02-07 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/20518#discussion_r166813909 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/KMeans.scala --- @@ -745,4 +763,27 @@ private[spark] class CosineDistanceMeasure

[GitHub] spark issue #20516: [SPARK-23343][CORE][TEST] Increase the exception test fo...

2018-02-07 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/20516 @srowen, if you don't set the spark.testing for true, the default value for spark.port.maxRetries is not 100, but 16, so in verifyServicePort function, actualPort should be <=

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20538 **[Test build #87181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87181/testReport)** for PR 20538 at commit

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20538 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/681/

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20538 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Pyt...

2018-02-07 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/20538 [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 without pandas. ## What changes were proposed in this pull request? This is a followup pr of #20487. When importing module

[GitHub] spark issue #20538: [SPARK-23319][TESTS][FOLLOWUP] Fix a test for Python 3 w...

2018-02-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20538 cc @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20537: [SPARK-23314][PYTHON] Add ambiguous=False when lo...

2018-02-07 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20537#discussion_r166811270 --- Diff: python/pyspark/sql/types.py --- @@ -1730,7 +1730,28 @@ def _check_series_convert_timestamps_internal(s, timezone): # TODO: handle

[GitHub] spark pull request #20537: [SPARK-23314][PYTHON] Add ambiguous=False when lo...

2018-02-07 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20537#discussion_r166810415 --- Diff: python/pyspark/sql/types.py --- @@ -1730,7 +1730,28 @@ def _check_series_convert_timestamps_internal(s, timezone): # TODO: handle

[GitHub] spark issue #19894: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-07 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19894 I'm going to backport this to 2.2 since it's a correctness bug. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166805463 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -228,14 +231,23 @@ private[spark] class EventLoggingListener(

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166805197 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -53,10 +53,21 @@ package object config { .booleanConf

[GitHub] spark pull request #20532: [SPARK-23353][CORE] Allow ExecutorMetricsUpdate e...

2018-02-07 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20532#discussion_r166805138 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -53,10 +53,21 @@ package object config { .booleanConf

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-02-07 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20373 Sounds good! No rush, I'll keep an eye out for the release --- - To unsubscribe, e-mail:

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-02-07 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20274 @liufengdb Can you verify this one? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-07 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20382 Sure, I will do it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/680/

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20534 **[Test build #87180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87180/testReport)** for PR 20534 at commit

[GitHub] spark pull request #20533: [SPARK-23300][TESTS][BRANCH-2.3] Prints out if Pa...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/20533 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20534 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20533: [SPARK-23300][TESTS][BRANCH-2.3] Prints out if Pandas an...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20533 Merged to branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-02-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20373 Yup, now the codes of branch "0.4.x" in cloudpickle is the same with the current PR. Was thinking of letting you know after 0.4.3. Please give me few days ... :-). ---

[GitHub] spark issue #20424: [Spark-23240][python] Better error message when extraneo...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20424 **[Test build #87179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87179/testReport)** for PR 20424 at commit

[GitHub] spark pull request #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSo...

2018-02-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20445 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20537 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20537 **[Test build #87178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87178/testReport)** for PR 20537 at commit

[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87178/ Test PASSed. ---

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20445 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87176/ Test PASSed. ---

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20445 **[Test build #87176 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87176/testReport)** for PR 20445 at commit

[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20537 **[Test build #87178 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87178/testReport)** for PR 20537 at commit

<    1   2   3   4   5   6   >