[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #88619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88619/testReport)** for PR 19222 at commit

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1776/

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20844: [SPARK-23707][SQL] Don't need shuffle exchange wi...

2018-03-26 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/20844#discussion_r177311524 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ConfigBehaviorSuite.scala --- @@ -39,7 +39,7 @@ class ConfigBehaviorSuite extends QueryTest with

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177306568 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java --- @@ -69,8 +64,9 @@ public long size() { * Fill this all with

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20906 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20906 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88611/ Test PASSed. ---

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20906 **[Test build #88611 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88611/testReport)** for PR 20906 at commit

[GitHub] spark pull request #20756: [SPARK-23593][SQL] Add interpreted execution for ...

2018-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20756#discussion_r177302845 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ObjectExpressionsSuite.scala --- @@ -68,6 +68,32 @@ class

[GitHub] spark issue #20756: [SPARK-23593][SQL] Add interpreted execution for Initial...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20756 **[Test build #88618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88618/testReport)** for PR 20756 at commit

[GitHub] spark issue #20756: [SPARK-23593][SQL] Add interpreted execution for Initial...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20756 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20756: [SPARK-23593][SQL] Add interpreted execution for Initial...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20756 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1775/

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20906 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20906 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88614/ Test FAILed. ---

[GitHub] spark pull request #20756: [SPARK-23593][SQL] Add interpreted execution for ...

2018-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20756#discussion_r177302386 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ObjectExpressionsSuite.scala --- @@ -68,6 +68,23 @@ class

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20906 **[Test build #88614 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88614/testReport)** for PR 20906 at commit

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1774/

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88617/testReport)** for PR 20784 at commit

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20753: [SPARK-23582][SQL] StaticInvoke should support interpret...

2018-03-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20753 @hvanhovell sorry for ping again --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20850 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1773/

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20850 **[Test build #88616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88616/testReport)** for PR 20850 at commit

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20797: [SPARK-23583][SQL] Invoke should support interpreted exe...

2018-03-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20797 @hvanhovell sorry for ping again --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177296492 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/OnHeapMemoryBlock.java --- @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20908: [SPARK-23672][PYTHON] Document support for nested...

2018-03-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20908#discussion_r177296271 --- Diff: python/pyspark/sql/tests.py --- @@ -3966,6 +3967,15 @@ def random_udf(v): random_udf = random_udf.asNondeterministic()

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177296342 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,161 @@ */ public static final int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177296162 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,161 @@ */ public static final int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177295079 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java --- @@ -89,6 +85,6 @@ public void set(int index, long value) {

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177295038 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java --- @@ -80,7 +76,7 @@ public void zeroOut() { public void

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r177294950 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java --- @@ -69,8 +64,9 @@ public long size() { * Fill this all

[GitHub] spark pull request #20908: [SPARK-23672][PYTHON] Document support for nested...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20908#discussion_r177294401 --- Diff: python/pyspark/sql/tests.py --- @@ -3966,6 +3967,15 @@ def random_udf(v): random_udf = random_udf.asNondeterministic()

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177293958 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -0,0 +1,164 @@ +/* + *

[GitHub] spark pull request #20908: [SPARK-23672][PYTHON] Document support for nested...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20908#discussion_r177293289 --- Diff: python/pyspark/sql/tests.py --- @@ -3806,6 +3806,7 @@ def test_timestamp_dst(self): not _have_pandas or not _have_pyarrow,

[GitHub] spark pull request #20784: [SPARK-23639][SQL]Obtain token before init metast...

2018-03-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20784#discussion_r177291761 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala --- @@ -77,6 +80,12 @@ private[hive] object

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20784 cc @jerryshao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177290859 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -0,0 +1,158 @@ +/* + *

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177290805 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkPod.scala --- @@ -14,17 +14,16 @@ * See the License

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88615/testReport)** for PR 20784 at commit

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1772/

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20906 **[Test build #88614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88614/testReport)** for PR 20906 at commit

[GitHub] spark pull request #20906: [SPARK-23561][SS] Pull continuous processing out ...

2018-03-26 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20906#discussion_r177288972 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousWriteExec.scala --- @@ -0,0 +1,122 @@ +/* + *

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1771/

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20906 All that should be left before removing the inheritance is to introduce a continuous version of the logical plan WriteToDataSourceV2. I wanted to do that in a separate PR, since there's already

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88613/testReport)** for PR 20784 at commit

[GitHub] spark pull request #20784: [SPARK-23639][SQL]Obtain token before init metast...

2018-03-26 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/20784#discussion_r177288126 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala --- @@ -121,6 +134,25 @@ private[hive]

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20784 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88612/ Test FAILed. ---

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88612 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88612/testReport)** for PR 20784 at commit

[GitHub] spark issue #20784: [SPARK-23639][SQL]Obtain token before init metastore cli...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20784 **[Test build #88612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88612/testReport)** for PR 20784 at commit

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1747/ ---

[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88604/ Test FAILed. ---

[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20893 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20893: [SPARK-23785][LAUNCHER] LauncherBackend doesn't check st...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20893 **[Test build #88604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88604/testReport)** for PR 20893 at commit

[GitHub] spark pull request #20906: [SPARK-23561][SS] Pull continuous processing out ...

2018-03-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20906#discussion_r177244722 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousWriteExec.scala --- @@ -0,0 +1,122 @@ +/* + *

[GitHub] spark pull request #20906: [SPARK-23561][SS] Pull continuous processing out ...

2018-03-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20906#discussion_r177245022 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousWriteExec.scala --- @@ -0,0 +1,122 @@ +/* + *

[GitHub] spark pull request #20906: [SPARK-23561][SS] Pull continuous processing out ...

2018-03-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20906#discussion_r177275489 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousWriteExec.scala --- @@ -0,0 +1,122 @@ +/* + *

[GitHub] spark pull request #20906: [SPARK-23561][SS] Pull continuous processing out ...

2018-03-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20906#discussion_r177243246 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousWriteExec.scala --- @@ -0,0 +1,122 @@ +/* + *

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20906 **[Test build #88611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88611/testReport)** for PR 20906 at commit

[GitHub] spark issue #20906: [SPARK-23561][SS] Pull continuous processing out of Writ...

2018-03-26 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20906 jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20858: [SPARK-23736][SQL] Extending the concat function ...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20858#discussion_r177279540 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -287,3 +289,152 @@ case class

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88610/ Test PASSed. ---

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 **[Test build #88610 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88610/testReport)** for PR 20910 at commit

[GitHub] spark pull request #20858: [SPARK-23736][SQL] Extending the concat function ...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20858#discussion_r177278880 --- Diff: python/pyspark/sql/functions.py --- @@ -1834,6 +1819,25 @@ def array_contains(col, value): return

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1770/

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1747/ ---

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20858: [SPARK-23736][SQL] Extending the concat function ...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20858#discussion_r177278671 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -699,3 +699,88 @@ abstract class

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177277920 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -0,0 +1,158 @@ +/* + * Licensed

[GitHub] spark issue #20900: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_u...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20900 The issue itself here (SPARK-23645) describes kwargs arguments support in both UDF and Pandas UDF on calling side. Seems not working but the fix looks going to be quite invasive and big. So, I

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177277876 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -0,0 +1,164 @@ +/* + * Licensed

[GitHub] spark issue #20858: [SPARK-23736][SQL] Extending the concat function to supp...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20858 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20858: [SPARK-23736][SQL] Extending the concat function to supp...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20858 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88605/ Test FAILed. ---

[GitHub] spark issue #20858: [SPARK-23736][SQL] Extending the concat function to supp...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20858 **[Test build #88605 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88605/testReport)** for PR 20858 at commit

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 **[Test build #88610 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88610/testReport)** for PR 20910 at commit

[GitHub] spark issue #20900: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_u...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20900 to be clear, I think both functions below ```python class F(object): def __call__(...): ... func = F() ``` ```python def naive_func(a,

[GitHub] spark pull request #20836: SPARK-23685 : Fix for the Spark Structured Stream...

2018-03-26 Thread sirishaSindri
Github user sirishaSindri commented on a diff in the pull request: https://github.com/apache/spark/pull/20836#discussion_r177276642 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala --- @@ -279,9 +279,8 @@ private[kafka010]

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1746/ ---

[GitHub] spark issue #20900: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_u...

2018-03-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20900 @icexelloss, yup ^ is correct. IIRC, we have some tests for normal udfs with callable objects and partial functions separately but seems the problem is in Pandas UDF. I think the fix itself

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88609/ Test PASSed. ---

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 **[Test build #88609 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88609/testReport)** for PR 20910 at commit

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1746/ ---

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1769/

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88608/ Test PASSed. ---

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 **[Test build #88608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88608/testReport)** for PR 20910 at commit

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177273537 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala --- @@ -180,20 +167,17 @@

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177273458 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkPod.scala --- @@ -14,17 +14,16 @@ * See the License for

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 **[Test build #88609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88609/testReport)** for PR 20910 at commit

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1768/

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20910 **[Test build #88608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88608/testReport)** for PR 20910 at commit

[GitHub] spark issue #20910: [SPARK-22839] [K8s] Refactor to unify driver and executo...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20910 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177267646 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala --- @@ -0,0 +1,137 @@

[GitHub] spark pull request #20910: [SPARK-22839] [K8s] Refactor to unify driver and ...

2018-03-26 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20910#discussion_r177266945 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -0,0 +1,158 @@ +/* + *

  1   2   3   4   >