[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-14 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/18798 @yanboliang I will update ASAP, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-14 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18798 @WeichenXu123 I left some minor comments, otherwise, LGTM. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133120700 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,593 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133120739 --- Diff: mllib/src/test/scala/org/apache/spark/ml/stat/SummarizerSuite.scala --- @@ -0,0 +1,619 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133109722 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,593 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133115762 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,593 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133115706 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,593 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133120087 --- Diff: mllib/src/test/scala/org/apache/spark/ml/stat/SummarizerSuite.scala --- @@ -0,0 +1,619 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133111870 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,593 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #18938: [SPARK-21363][SQL] Prevent name duplication in (global/l...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18938 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18938: [SPARK-21363][SQL] Prevent name duplication in (global/l...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18938 **[Test build #80658 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80658/testReport)** for PR 18938 at commit

[GitHub] spark issue #18938: [SPARK-21363][SQL] Prevent name duplication in (global/l...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18938 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80658/ Test FAILed. ---

[GitHub] spark issue #18938: [SPARK-21363][SQL] Prevent name duplication in (global/l...

2017-08-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18938 IMHO, other database(-like) systems prevent this name duplication in temporary table/view, so it seems we might do so along with them. ``` // postgresql case postgres=# create table s0

[GitHub] spark pull request #18093: [WIP][SPARK-20774][SQL] Cancel all jobs when Quer...

2017-08-14 Thread liyichao
Github user liyichao closed the pull request at: https://github.com/apache/spark/pull/18093 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18938: [SPARK-21363][SQL] Prevent name duplication in (global/l...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18938 **[Test build #80658 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80658/testReport)** for PR 18938 at commit

[GitHub] spark issue #18093: [WIP][SPARK-20774][SQL] Cancel all jobs when QueryExecti...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18093 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133119397 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,593 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #18934: [SPARK-21721][SQL] Clear FileSystem deleteOnExit cache w...

2017-08-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18934 Tried to merge to Spark 2.1, but I hit a conflict. Thus, please let me know if anybody wants it in 2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #18934: [SPARK-21721][SQL] Clear FileSystem deleteOnExit ...

2017-08-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18934 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18934: [SPARK-21721][SQL] Clear FileSystem deleteOnExit cache w...

2017-08-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18934 Thanks! Merging to master/2.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-08-14 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/18902 Eh, I meant that it may be possible to get the mean values purely using DataFrame API. (convert missingValue/NaN to null) in one pass, so we may need to check the performance comparison. But I guess

[GitHub] spark issue #18934: [SPARK-21721][SQL] Clear FileSystem deleteOnExit cache w...

2017-08-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18934 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18946: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18946 LGTM except two minor comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18946: [SPARK-19471][SQL]AggregationIterator does not in...

2017-08-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18946#discussion_r133118690 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -573,4 +578,47 @@ class DataFrameAggregateSuite extends

[GitHub] spark issue #18946: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18946 **[Test build #80657 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80657/testReport)** for PR 18946 at commit

[GitHub] spark issue #18946: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18946 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #18946: [SPARK-19471][SQL]AggregationIterator does not in...

2017-08-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18946#discussion_r133118490 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -17,13 +17,18 @@ package org.apache.spark.sql

[GitHub] spark issue #18922: [SPARK-21701][CORE] Enable RPC client to use SO_RCVBUF, ...

2017-08-14 Thread neoremind
Github user neoremind commented on the issue: https://github.com/apache/spark/pull/18922 @zsxwing Would you mind verifying the patch for me? I notice you have contributed to rpc module in spark. Many thanks! --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18926 LGTM except for the comment above. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18926: [SPARK-21712] [PySpark] Clarify type error for Co...

2017-08-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18926#discussion_r133116060 --- Diff: python/pyspark/sql/tests.py --- @@ -1220,6 +1220,13 @@ def test_rand_functions(self): rndn2 = df.select('key',

[GitHub] spark pull request #18930: [SPARK-21677][SQL] json_tuple throws NullPointExc...

2017-08-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18930#discussion_r133117021 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -359,14 +359,14 @@ case class

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-14 Thread aray
Github user aray commented on the issue: https://github.com/apache/spark/pull/18818 @viirya @gatorsmile I have addressed your comments, could you take another look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #18818: [SPARK-21110][SQL] Structs, arrays, and other ord...

2017-08-14 Thread aray
Github user aray commented on a diff in the pull request: https://github.com/apache/spark/pull/18818#discussion_r133116720 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -465,7 +475,7 @@ abstract class BinaryComparison

[GitHub] spark issue #18930: [SPARK-21677][SQL] json_tuple throws NullPointException ...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18930 **[Test build #80656 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80656/testReport)** for PR 18930 at commit

[GitHub] spark pull request #18930: [SPARK-21677][SQL] json_tuple throws NullPointExc...

2017-08-14 Thread jmchung
Github user jmchung commented on a diff in the pull request: https://github.com/apache/spark/pull/18930#discussion_r133116488 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2034,4 +2034,13 @@ class JsonSuite extends

[GitHub] spark issue #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18943 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18943 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80652/ Test PASSed. ---

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133116269 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,593 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18943 **[Test build #80652 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80652/testReport)** for PR 18943 at commit

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133115997 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,593 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18798 @WeichenXu123 Thanks! Looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18798#discussion_r133114092 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,593 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #18936: [SPARK-21688][ML][MLLIB] make native BLAS the first choi...

2017-08-14 Thread VinceShieh
Github user VinceShieh commented on the issue: https://github.com/apache/spark/pull/18936 Yes, they are not the only place, but we only tested on the dense dataset and got the performance data shown above. We are conservative on sparse data, so keep the sparse path the way it was.

[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-14 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/18798 @viirya Sure! comment updated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18798 Sorry can we make the performance data clear? Currently it doesn't say what the unit of the numbers is. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #18944: [SPARK-21732][SQL]Lazily init hive metastore client

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18944 **[Test build #80655 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80655/testReport)** for PR 18944 at commit

[GitHub] spark issue #18944: [SPARK-21732][SQL]Lazily init hive metastore client

2017-08-14 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18944 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #18930: [SPARK-21677][SQL] json_tuple throws NullPointExc...

2017-08-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18930#discussion_r133112441 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2034,4 +2034,13 @@ class JsonSuite extends

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80654 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80654/testReport)** for PR 18931 at commit

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18902 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18902 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80653/ Test PASSed. ---

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18902 **[Test build #80653 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80653/testReport)** for PR 18902 at commit

[GitHub] spark issue #18905: [SPARK-21660][YARN][Shuffle] Yarn ShuffleService failed ...

2017-08-14 Thread LiShuMing
Github user LiShuMing commented on the issue: https://github.com/apache/spark/pull/18905 @jerryshao Thanks for your replies! I will do such things then: 1. "it is good to change to other directories (is yarn internally relying on it)?" I think the recovery path(local

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-08-14 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/17014 ping @MLnick ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18946: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-14 Thread DonnyZone
Github user DonnyZone commented on the issue: https://github.com/apache/spark/pull/18946 @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18944: [SPARK-21732][SQL]Lazily init hive metastore client

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18944 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80651/ Test FAILed. ---

[GitHub] spark issue #18944: [SPARK-21732][SQL]Lazily init hive metastore client

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18944 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18944: [SPARK-21732][SQL]Lazily init hive metastore client

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18944 **[Test build #80651 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80651/testReport)** for PR 18944 at commit

[GitHub] spark issue #18946: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18946 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18946: [SPARK-19471][SQL]AggregationIterator does not in...

2017-08-14 Thread DonnyZone
GitHub user DonnyZone opened a pull request: https://github.com/apache/spark/pull/18946 [SPARK-19471][SQL]AggregationIterator does not initialize the generated result projection before using it ## What changes were proposed in this pull request? This is a follow-up PR that

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18902 **[Test build #80653 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80653/testReport)** for PR 18902 at commit

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-08-14 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r133102376 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -35,11 +38,14 @@ import

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-08-14 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r133102774 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -117,17 +123,38 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-08-14 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r133104552 --- Diff: docs/monitoring.md --- @@ -220,6 +220,13 @@ The history server can be configured as follows: Number of threads that will be used by

[GitHub] spark issue #18944: [SPARK-21732][SQL]Lazily init hive metastore client

2017-08-14 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/18944 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18934: [SPARK-21721][SQL] Clear FileSystem deleteOnExit cache w...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18934 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80649/ Test PASSed. ---

[GitHub] spark issue #18934: [SPARK-21721][SQL] Clear FileSystem deleteOnExit cache w...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18934 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18934: [SPARK-21721][SQL] Clear FileSystem deleteOnExit cache w...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18934 **[Test build #80649 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80649/testReport)** for PR 18934 at commit

[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2017-08-14 Thread KevinZwx
Github user KevinZwx commented on the issue: https://github.com/apache/spark/pull/18903 I added some tests in `DataFrameTimeWindowingSuite`, please have a look @brkyvz --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #18945: Add option to convert nullable int columns to float colu...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18945 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18945: Add option to convert nullable int columns to flo...

2017-08-14 Thread logannc
GitHub user logannc opened a pull request: https://github.com/apache/spark/pull/18945 Add option to convert nullable int columns to float columns in toPand… …as to prevent needless Exceptions during routine use. Add the `strict=True` kwarg to DataFrame.toPandas to allow

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18818 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80647/ Test PASSed. ---

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18818 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18818 **[Test build #80647 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80647/testReport)** for PR 18818 at commit

[GitHub] spark issue #18942: [BACKPORT-2.1][SPARK-19372][SQL] Fix throwing a Java exc...

2017-08-14 Thread poplav
Github user poplav commented on the issue: https://github.com/apache/spark/pull/18942 Looking back at this. I simply cherry picked the commit from the branch, there appears to be more to this backport. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80648/ Test FAILed. ---

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80648 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80648/testReport)** for PR 18931 at commit

[GitHub] spark issue #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18943 **[Test build #80652 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80652/testReport)** for PR 18943 at commit

[GitHub] spark issue #18760: [SPARK-21560][Core] Add hold mode for the LiveListenerBu...

2017-08-14 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/18760 @jiangxb1987 Hi xingbo, can you give me some advise about this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18943 Argh, forgot to update the tests... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18944: [SPARK-21732][SQL]Lazily init hive metastore client

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18944 **[Test build #80651 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80651/testReport)** for PR 18944 at commit

[GitHub] spark pull request #18944: [SPARK-21732][SQL]Lazily init hive metastore clie...

2017-08-14 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/18944#discussion_r133096415 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/HiveMetastoreLazyInitializationSuite.scala --- @@ -0,0 +1,57 @@ +/* ---

[GitHub] spark issue #18944: [SPARK-21732][SQL]Lazily init hive metastore client

2017-08-14 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18944 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #18944: [SPARK-21732][SQL]Lazily init hive metastore clie...

2017-08-14 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/18944 [SPARK-21732][SQL]Lazily init hive metastore client ## What changes were proposed in this pull request? This PR changes the codes to lazily init hive metastore client so that we can

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18664 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18664 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80646/ Test PASSed. ---

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18664 **[Test build #80646 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80646/testReport)** for PR 18664 at commit

[GitHub] spark issue #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18943 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18943 **[Test build #80650 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80650/testReport)** for PR 18943 at commit

[GitHub] spark issue #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18943 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80650/ Test FAILed. ---

[GitHub] spark issue #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18943 **[Test build #80650 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80650/testReport)** for PR 18943 at commit

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-08-14 Thread poplav
Github user poplav commented on the issue: https://github.com/apache/spark/pull/17087 Hi, All. I am trying to get this included into Spark 2.1.1. I opened a PR https://github.com/apache/spark/pull/18942. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/18943 [SPARK-21731][build] Upgrade scalastyle to 0.9. This version fixes a few issues in the import order checker; it provides better error messages, and detects more improper ordering (thus the need

[GitHub] spark issue #18943: [SPARK-21731][build] Upgrade scalastyle to 0.9.

2017-08-14 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18943 Will also test with maven once sbt tests pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18934: [SPARK-21721][SQL] Clear FileSystem deleteOnExit cache w...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18934 **[Test build #80649 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80649/testReport)** for PR 18934 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80648 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80648/testReport)** for PR 18931 at commit

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18874 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18874 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80645/ Test PASSed. ---

[GitHub] spark issue #18942: [BACKPORT-2.1][SPARK-19372][SQL] Fix throwing a Java exc...

2017-08-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18942 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

  1   2   3   4   5   >