[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20864 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88537/ Test FAILed. ---

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20864 **[Test build #88537 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88537/testReport)** for PR 20864 at commit [`b177bf4`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20864 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20860: [SPARK-23743][SQL] Changed a comparison logic fro...

2018-03-23 Thread jongyoul
Github user jongyoul commented on a diff in the pull request: https://github.com/apache/spark/pull/20860#discussion_r176652812 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala --- @@ -179,7 +179,7 @@ private[hive] class IsolatedClientLo

[GitHub] spark pull request #20860: [SPARK-23743][SQL] Changed a comparison logic fro...

2018-03-23 Thread jongyoul
Github user jongyoul commented on a diff in the pull request: https://github.com/apache/spark/pull/20860#discussion_r176654007 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala --- @@ -179,7 +179,7 @@ private[hive] class IsolatedClientLo

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1719/ Tes

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20860 **[Test build #88538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88538/testReport)** for PR 20860 at commit [`2ea9b7a`](https://github.com/apache/spark/commit/2e

[GitHub] spark pull request #20836: SPARK-23685 : Fix for the Spark Structured Stream...

2018-03-23 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20836#discussion_r176656905 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala --- @@ -279,9 +279,8 @@ private[kafka010] case

[GitHub] spark issue #20836: SPARK-23685 : Fix for the Spark Structured Streaming Kaf...

2018-03-23 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/20836 I don't see the problem. You see an exception which tells exactly what can be done: > If you don't want your streaming query to fail on such cases, set the source option "failOnDataLo

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-23 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r176660685 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -353,6 +353,13 @@ object SQLConf { .booleanConf .cr

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/20864 @felixcheung Thanks. The test result is 0 failed, what is meaning of > This patch fails due to an unknown error code, -9 ? what should i do to fix the test fails? --- ---

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20885 **[Test build #88539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88539/testReport)** for PR 20885 at commit [`d632706`](https://github.com/apache/spark/commit/d6

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20885 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88539/ Test FAILed. ---

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20885 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20885 **[Test build #88539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88539/testReport)** for PR 20885 at commit [`d632706`](https://github.com/apache/spark/commit/d

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r176676405 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -353,6 +353,13 @@ object SQLConf { .booleanConf

[GitHub] spark issue #20541: [SPARK-23356][SQL]Pushes Project to both sides of Union ...

2018-03-23 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/20541 oh, I see, I fallback to the modification of the non-deterministic expression, and to keep the newly added test cases for a+1 and a+b, can you? --- --

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20860 **[Test build #88538 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88538/testReport)** for PR 20860 at commit [`2ea9b7a`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88538/ Test PASSed. ---

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20811: [SPARK-23668][K8S] Add config option for passing ...

2018-03-23 Thread andrusha
Github user andrusha commented on a diff in the pull request: https://github.com/apache/spark/pull/20811#discussion_r176687092 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -108,6 +109,8 @@ private

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20811 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1701/ --- --

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20811 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20811 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1720/ Tes

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread Lemonjing
Github user Lemonjing commented on the issue: https://github.com/apache/spark/pull/20889 @felixcheung hi,In my reading process,I found another md file typos and fix it. Do I need to close this pr and merge it into one commit and give a new pr? Thanks. --- -

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20811 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1701/ ---

[GitHub] spark issue #20877: [SPARK-23765][SQL] Supports custom line separator for js...

2018-03-23 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20877 > Does that fix actual usecases? I see the following use cases: 1. Jsons coming usually from embedded systems have not-standard separators (invisible in some cases). It is very conv

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20885 **[Test build #88540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88540/testReport)** for PR 20885 at commit [`bbff402`](https://github.com/apache/spark/commit/bb

[GitHub] spark issue #20879: [MINOR][R] Fix R lint failure

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20879 Yup, we are running old lint in PR builders and those are on the newer one. --- - To unsubscribe, e-mail: reviews-unsubscr...

[GitHub] spark issue #20879: [MINOR][R] Fix R lint failure

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20879 Merged to master and branch-2.3. Thanks for reviewing this @shaneknapp and @felixcheung. --- - To unsubscribe, e-mai

[GitHub] spark issue #20880: [SPARK-23769][Core]Remove comments that unnecessarily di...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20880 Merged to master and branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20880: [SPARK-23769][Core]Remove comments that unnecessa...

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20880 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20879: [MINOR][R] Fix R lint failure

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20879 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20889 I think you can add all other typos into here. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additio

[GitHub] spark issue #20884: [SPARK-23773][SQL] JacksonGenerator does not include key...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20884 Shall we add a configuration to control its behaviour if this is something we need to support? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20884: [SPARK-23773][SQL] JacksonGenerator does not incl...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20884#discussion_r176715906 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonGeneratorSuite.scala --- @@ -56,7 +56,7 @@ class JacksonGeneratorSuite

[GitHub] spark issue #20877: [SPARK-23765][SQL] Supports custom line separator for js...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20877 Yup, yup. I don't object for now. Shall we merge this one first and talk more about it in your PR? I believe this PR itself proposes a complete option and I saw many the requests for this fe

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20885 @cloud-fan and @hvanhovell. Do you think we need the flexible option for line separator? https://github.com/apache/spark/blob/bbff40206e6871ea9ab035e7a8876f495bdf3d90/sql/catalyst/src/m

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread Lemonjing
Github user Lemonjing commented on the issue: https://github.com/apache/spark/pull/20889 @HyukjinKwon Thanks a lot, could u help me to review these commits if u have time。 --- - To unsubscribe, e-mail: reviews-uns

[GitHub] spark issue #20877: [SPARK-23765][SQL] Supports custom line separator for js...

2018-03-23 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20877 I have only one concern: if we merge this PR, we close the possibility for changing format of `lineSep` and future extensions. Your changes allow any sequence of chars. It is not clear for me, how w

[GitHub] spark pull request #19881: [SPARK-22683][CORE] Add a fullExecutorAllocationD...

2018-03-23 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19881#discussion_r176727530 --- Diff: core/src/test/scala/org/apache/spark/ExecutorAllocationManagerSuite.scala --- @@ -145,6 +145,39 @@ class ExecutorAllocationManagerSuite

[GitHub] spark issue #19881: [SPARK-22683][CORE] Add a fullExecutorAllocationDivisor ...

2018-03-23 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19881 +1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apach

[GitHub] spark pull request #20800: [SPARK-23627][SQL] Provide isEmpty in Dataset

2018-03-23 Thread goungoun
Github user goungoun commented on a diff in the pull request: https://github.com/apache/spark/pull/20800#discussion_r176728379 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -511,6 +511,14 @@ class Dataset[T] private[sql]( */ def isLocal:

[GitHub] spark issue #20877: [SPARK-23765][SQL] Supports custom line separator for js...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20877 I think we are fine to change the behaviour of `lineSep` before the release .. --- - To unsubscribe, e-mail: reviews-unsubsc

[GitHub] spark issue #20888: [SPARK-23775][TEST] DataFrameRangeSuite should wait for ...

2018-03-23 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20888 ah ok, yes when run in isolation, the stage will be 0, so your change makes sense. But that is not what is making it flaky in a full test run --- --

[GitHub] spark issue #20883: [SPARK-23759][UI] Unable to bind Spark UI to specific ho...

2018-03-23 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20883 please update the description moving > Fixes SPARK-23759 by moving connector.start() after connector.setHost() after the title (What changes...) and please remove the sentence:

[GitHub] spark pull request #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and Uns...

2018-03-23 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/20890 [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSorter related classes use MemoryBlock ## What changes were proposed in this pull request? Waiting for merging #19222 This PR tri

[GitHub] spark issue #20883: [SPARK-23759][UI] Unable to bind Spark UI to specific ho...

2018-03-23 Thread felixalbani
Github user felixalbani commented on the issue: https://github.com/apache/spark/pull/20883 @mgaido91 Thanks, I updated description with your suggestions --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20890 **[Test build #88541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88541/testReport)** for PR 20890 at commit [`fce58f8`](https://github.com/apache/spark/commit/fc

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1721/ Tes

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20883: [SPARK-23759][UI] Unable to bind Spark UI to specific ho...

2018-03-23 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20883 Thanks, LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20890 **[Test build #88541 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88541/testReport)** for PR 20890 at commit [`fce58f8`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88541/ Test FAILed. ---

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20717: [SPARK-23564][SQL] Add isNotNull check for left a...

2018-03-23 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20717#discussion_r176750470 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -341,6 +341,26 @@ case class Join(

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20717 **[Test build #88542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88542/testReport)** for PR 20717 at commit [`5cadd86`](https://github.com/apache/spark/commit/5c

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20717 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1722/ Tes

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20717 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20818: [SPARK-23675][WEB-UI]Title add spark logo, use sp...

2018-03-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20818#discussion_r176753823 --- Diff: core/src/main/scala/org/apache/spark/ui/UIUtils.scala --- @@ -265,6 +266,7 @@ private[spark] object UIUtils extends Logging {

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20885 **[Test build #88540 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88540/testReport)** for PR 20885 at commit [`bbff402`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20885 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20885 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88540/ Test PASSed. ---

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1723/ Tes

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20890 **[Test build #88543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88543/testReport)** for PR 20890 at commit [`33b692a`](https://github.com/apache/spark/commit/33

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20890 **[Test build #88543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88543/testReport)** for PR 20890 at commit [`33b692a`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88543/ Test FAILed. ---

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r176762766 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -353,6 +353,13 @@ object SQLConf { .booleanConf

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20811 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1724/ Tes

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20811 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20811 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1705/ --- --

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20890 **[Test build #88544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88544/testReport)** for PR 20890 at commit [`4a0a5e3`](https://github.com/apache/spark/commit/4a

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1725/ Tes

[GitHub] spark issue #20890: [WIP][SPARK-23779][SQL] TaskMemoryManager and UnsafeSort...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20890 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20811 LGTM. @foxish. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20811 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1705/ ---

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-03-23 Thread cclauss
Github user cclauss commented on the issue: https://github.com/apache/spark/pull/20838 @HyukjinKwon Was there something more to do on this one? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For a

[GitHub] spark pull request #20884: [SPARK-23773][SQL] JacksonGenerator does not incl...

2018-03-23 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/20884#discussion_r176796998 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonGeneratorSuite.scala --- @@ -56,7 +56,7 @@ class JacksonGeneratorSuit

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread liufengdb
Github user liufengdb commented on the issue: https://github.com/apache/spark/pull/20864 I thought the directory is also created from this line: https://github.com/apache/spark/blob/master/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java#L143

[GitHub] spark issue #20883: [SPARK-23759][UI] Unable to bind Spark UI to specific ho...

2018-03-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20883 Merging to master, 2.3, 2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark pull request #20883: [SPARK-23759][UI] Unable to bind Spark UI to spec...

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20883 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20717 **[Test build #88542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88542/testReport)** for PR 20717 at commit [`5cadd86`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20717 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20717 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88542/ Test PASSed. ---

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread gerashegalov
Github user gerashegalov commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176816848 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite ex

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176817667 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite extends

[GitHub] spark pull request #20885: [SPARK-23724][SPARK-23765][SQL] Line separator fo...

2018-03-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20885#discussion_r176820312 --- Diff: python/pyspark/sql/readwriter.py --- @@ -770,12 +773,15 @@ def json(self, path, mode=None, compression=None, dateFormat=None, timestampForm

[GitHub] spark pull request #20885: [SPARK-23724][SPARK-23765][SQL] Line separator fo...

2018-03-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20885#discussion_r176821110 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -85,6 +85,38 @@ private[sql] class JSONOptions(

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-03-23 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r176821060 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -86,7 +88,80 @@ private[util] sealed trait BaseReadWrite { } /

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-03-23 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r176821909 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -86,7 +88,80 @@ private[util] sealed trait BaseReadWrite { } /

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread gerashegalov
Github user gerashegalov commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176822894 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite ex

[GitHub] spark pull request #20891: [SPARK-23782][CORE][UI] SHS should list only appl...

2018-03-23 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/20891 [SPARK-23782][CORE][UI] SHS should list only applications with read permissions ## What changes were proposed in this pull request? Before the PR, all applications were returned to all us

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20891 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20891 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1726/ Tes

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176823881 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite extends

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20891 **[Test build #88545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88545/testReport)** for PR 20891 at commit [`bc87945`](https://github.com/apache/spark/commit/bc

[GitHub] spark pull request #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet st...

2018-03-23 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18982#discussion_r176825209 --- Diff: python/pyspark/ml/wrapper.py --- @@ -118,11 +118,18 @@ def _transfer_params_to_java(self): """ Transforms the embedde

  1   2   3   >