[GitHub] spark issue #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20892 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark....

2018-03-23 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20892#discussion_r176829827 --- Diff: python/pyspark/sql/functions.py --- @@ -28,10 +27,10 @@ from pyspark import since, SparkContext from pyspark.rdd import

[GitHub] spark issue #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql

2018-03-23 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20892 @HyukjinKwon and @ueshin I tried to be pretty conservative and only remove imports that were obviously not being used. ---

[GitHub] spark pull request #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet st...

2018-03-23 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/18982#discussion_r176828963 --- Diff: python/pyspark/ml/wrapper.py --- @@ -118,11 +118,18 @@ def _transfer_params_to_java(self): """ Transforms the embedded

[GitHub] spark issue #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to Spark ...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to Spark ...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1727/

[GitHub] spark pull request #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark....

2018-03-23 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/20892 [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql ## What changes were proposed in this pull request? This cleans up unused imports, mainly from pyspark.sql module. Added a note

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20891 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88545/ Test FAILed. ---

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20891 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20891 **[Test build #88545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88545/testReport)** for PR 20891 at commit

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176827527 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite extends

[GitHub] spark issue #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to Spark ...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19876 **[Test build #88546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88546/testReport)** for PR 19876 at commit

[GitHub] spark issue #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to Spark ...

2018-03-23 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19876 LGTM pending Jenkins will merge. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-03-23 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r176826923 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -86,7 +88,80 @@ private[util] sealed trait BaseReadWrite { }

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread gerashegalov
Github user gerashegalov commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176825696 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite

[GitHub] spark pull request #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet st...

2018-03-23 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18982#discussion_r176825209 --- Diff: python/pyspark/ml/wrapper.py --- @@ -118,11 +118,18 @@ def _transfer_params_to_java(self): """ Transforms the

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176823881 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite extends

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20891 **[Test build #88545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88545/testReport)** for PR 20891 at commit

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20891 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20891 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1726/

[GitHub] spark pull request #20891: [SPARK-23782][CORE][UI] SHS should list only appl...

2018-03-23 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/20891 [SPARK-23782][CORE][UI] SHS should list only applications with read permissions ## What changes were proposed in this pull request? Before the PR, all applications were returned to all

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread gerashegalov
Github user gerashegalov commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176822894 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-03-23 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r176821909 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -86,7 +88,80 @@ private[util] sealed trait BaseReadWrite { }

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-03-23 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r176821060 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -86,7 +88,80 @@ private[util] sealed trait BaseReadWrite { }

[GitHub] spark pull request #20885: [SPARK-23724][SPARK-23765][SQL] Line separator fo...

2018-03-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20885#discussion_r176821110 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -85,6 +85,38 @@ private[sql] class JSONOptions(

[GitHub] spark pull request #20885: [SPARK-23724][SPARK-23765][SQL] Line separator fo...

2018-03-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20885#discussion_r176820312 --- Diff: python/pyspark/sql/readwriter.py --- @@ -770,12 +773,15 @@ def json(self, path, mode=None, compression=None, dateFormat=None, timestampForm

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176817667 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite extends

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread gerashegalov
Github user gerashegalov commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176816848 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20717 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88542/ Test PASSed. ---

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20717 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20717 **[Test build #88542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88542/testReport)** for PR 20717 at commit

[GitHub] spark pull request #20883: [SPARK-23759][UI] Unable to bind Spark UI to spec...

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20883 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20883: [SPARK-23759][UI] Unable to bind Spark UI to specific ho...

2018-03-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20883 Merging to master, 2.3, 2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread liufengdb
Github user liufengdb commented on the issue: https://github.com/apache/spark/pull/20864 I thought the directory is also created from this line:

[GitHub] spark pull request #20884: [SPARK-23773][SQL] JacksonGenerator does not incl...

2018-03-23 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/20884#discussion_r176796998 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonGeneratorSuite.scala --- @@ -56,7 +56,7 @@ class

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-03-23 Thread cclauss
Github user cclauss commented on the issue: https://github.com/apache/spark/pull/20838 @HyukjinKwon Was there something more to do on this one? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20811 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1705/ ---

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20811 LGTM. @foxish. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1725/

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20890 **[Test build #88544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88544/testReport)** for PR 20890 at commit

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20811 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1705/ ---

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20811 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20811 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1724/

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r176762766 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -353,6 +353,13 @@ object SQLConf { .booleanConf

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88543/ Test FAILed. ---

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20890 **[Test build #88543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88543/testReport)** for PR 20890 at commit

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20890 **[Test build #88543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88543/testReport)** for PR 20890 at commit

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1723/

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20885 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20885 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88540/ Test PASSed. ---

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20885 **[Test build #88540 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88540/testReport)** for PR 20885 at commit

[GitHub] spark pull request #20818: [SPARK-23675][WEB-UI]Title add spark logo, use sp...

2018-03-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20818#discussion_r176753823 --- Diff: core/src/main/scala/org/apache/spark/ui/UIUtils.scala --- @@ -265,6 +266,7 @@ private[spark] object UIUtils extends Logging {

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20717 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20717 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1722/

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20717 **[Test build #88542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88542/testReport)** for PR 20717 at commit

[GitHub] spark pull request #20717: [SPARK-23564][SQL] Add isNotNull check for left a...

2018-03-23 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20717#discussion_r176750470 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -341,6 +341,26 @@ case class Join(

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20890 **[Test build #88541 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88541/testReport)** for PR 20890 at commit

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88541/ Test FAILed. ---

[GitHub] spark issue #20883: [SPARK-23759][UI] Unable to bind Spark UI to specific ho...

2018-03-23 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20883 Thanks, LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20890 **[Test build #88541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88541/testReport)** for PR 20890 at commit

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1721/

[GitHub] spark issue #20883: [SPARK-23759][UI] Unable to bind Spark UI to specific ho...

2018-03-23 Thread felixalbani
Github user felixalbani commented on the issue: https://github.com/apache/spark/pull/20883 @mgaido91 Thanks, I updated description with your suggestions --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and Uns...

2018-03-23 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/20890 [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSorter related classes use MemoryBlock ## What changes were proposed in this pull request? Waiting for merging #19222 This PR

[GitHub] spark issue #20883: [SPARK-23759][UI] Unable to bind Spark UI to specific ho...

2018-03-23 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20883 please update the description moving > Fixes SPARK-23759 by moving connector.start() after connector.setHost() after the title (What changes...) and please remove the sentence:

[GitHub] spark issue #20888: [SPARK-23775][TEST] DataFrameRangeSuite should wait for ...

2018-03-23 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20888 ah ok, yes when run in isolation, the stage will be 0, so your change makes sense. But that is not what is making it flaky in a full test run ---

[GitHub] spark issue #20877: [SPARK-23765][SQL] Supports custom line separator for js...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20877 I think we are fine to change the behaviour of `lineSep` before the release .. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20800: [SPARK-23627][SQL] Provide isEmpty in Dataset

2018-03-23 Thread goungoun
Github user goungoun commented on a diff in the pull request: https://github.com/apache/spark/pull/20800#discussion_r176728379 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -511,6 +511,14 @@ class Dataset[T] private[sql]( */ def isLocal:

[GitHub] spark issue #19881: [SPARK-22683][CORE] Add a fullExecutorAllocationDivisor ...

2018-03-23 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19881 +1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19881: [SPARK-22683][CORE] Add a fullExecutorAllocationD...

2018-03-23 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19881#discussion_r176727530 --- Diff: core/src/test/scala/org/apache/spark/ExecutorAllocationManagerSuite.scala --- @@ -145,6 +145,39 @@ class ExecutorAllocationManagerSuite

[GitHub] spark issue #20877: [SPARK-23765][SQL] Supports custom line separator for js...

2018-03-23 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20877 I have only one concern: if we merge this PR, we close the possibility for changing format of `lineSep` and future extensions. Your changes allow any sequence of chars. It is not clear for me, how

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread Lemonjing
Github user Lemonjing commented on the issue: https://github.com/apache/spark/pull/20889 @HyukjinKwon Thanks a lot, could u help me to review these commits if u have time。 --- - To unsubscribe, e-mail:

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20885 @cloud-fan and @hvanhovell. Do you think we need the flexible option for line separator?

[GitHub] spark issue #20877: [SPARK-23765][SQL] Supports custom line separator for js...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20877 Yup, yup. I don't object for now. Shall we merge this one first and talk more about it in your PR? I believe this PR itself proposes a complete option and I saw many the requests for this

[GitHub] spark pull request #20884: [SPARK-23773][SQL] JacksonGenerator does not incl...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20884#discussion_r176715906 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonGeneratorSuite.scala --- @@ -56,7 +56,7 @@ class JacksonGeneratorSuite

[GitHub] spark issue #20884: [SPARK-23773][SQL] JacksonGenerator does not include key...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20884 Shall we add a configuration to control its behaviour if this is something we need to support? --- - To unsubscribe,

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20889 I think you can add all other typos into here. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20879: [MINOR][R] Fix R lint failure

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20879 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20880: [SPARK-23769][Core]Remove comments that unnecessa...

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20880 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20880: [SPARK-23769][Core]Remove comments that unnecessarily di...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20880 Merged to master and branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20879: [MINOR][R] Fix R lint failure

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20879 Merged to master and branch-2.3. Thanks for reviewing this @shaneknapp and @felixcheung. --- - To unsubscribe,

[GitHub] spark issue #20879: [MINOR][R] Fix R lint failure

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20879 Yup, we are running old lint in PR builders and those are on the newer one. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20885 **[Test build #88540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88540/testReport)** for PR 20885 at commit

[GitHub] spark issue #20877: [SPARK-23765][SQL] Supports custom line separator for js...

2018-03-23 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20877 > Does that fix actual usecases? I see the following use cases: 1. Jsons coming usually from embedded systems have not-standard separators (invisible in some cases). It is very

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20811 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1701/ ---

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread Lemonjing
Github user Lemonjing commented on the issue: https://github.com/apache/spark/pull/20889 @felixcheung hi,In my reading process,I found another md file typos and fix it. Do I need to close this pr and merge it into one commit and give a new pr? Thanks. ---

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20811 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1720/

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20811 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20811 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1701/ ---

[GitHub] spark pull request #20811: [SPARK-23668][K8S] Add config option for passing ...

2018-03-23 Thread andrusha
Github user andrusha commented on a diff in the pull request: https://github.com/apache/spark/pull/20811#discussion_r176687092 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -108,6 +109,8 @@

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88538/ Test PASSed. ---

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20860 **[Test build #88538 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88538/testReport)** for PR 20860 at commit

[GitHub] spark issue #20541: [SPARK-23356][SQL]Pushes Project to both sides of Union ...

2018-03-23 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/20541 oh, I see, I fallback to the modification of the non-deterministic expression, and to keep the newly added test cases for a+1 and a+b, can you? ---

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r176676405 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -353,6 +353,13 @@ object SQLConf { .booleanConf

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20885 **[Test build #88539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88539/testReport)** for PR 20885 at commit

<    1   2   3   >