[GitHub] spark pull request #20697: [SPARK-23010][k8s] Initial checkin of k8s integra...

2018-05-29 Thread ssuchter
Github user ssuchter commented on a diff in the pull request: https://github.com/apache/spark/pull/20697#discussion_r191485369 --- Diff: resource-managers/kubernetes/integration-tests/pom.xml --- @@ -0,0 +1,230 @@ + + +http://maven.apache.org/POM/4.0.0;

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 **[Test build #91264 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91264/testReport)** for PR 20697 at commit

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91264/ Test FAILed. ---

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3674/

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 **[Test build #91264 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91264/testReport)** for PR 20697 at commit

[GitHub] spark pull request #20697: [SPARK-23010][k8s] Initial checkin of k8s integra...

2018-05-29 Thread ssuchter
Github user ssuchter commented on a diff in the pull request: https://github.com/apache/spark/pull/20697#discussion_r191483976 --- Diff: resource-managers/kubernetes/integration-tests/pom.xml --- @@ -0,0 +1,230 @@ + + +http://maven.apache.org/POM/4.0.0;

[GitHub] spark pull request #20697: [SPARK-23010][k8s] Initial checkin of k8s integra...

2018-05-29 Thread ssuchter
Github user ssuchter commented on a diff in the pull request: https://github.com/apache/spark/pull/20697#discussion_r191483921 --- Diff: resource-managers/kubernetes/integration-tests/pom.xml --- @@ -0,0 +1,230 @@ + + +http://maven.apache.org/POM/4.0.0;

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191483040 --- Diff: python/pyspark/sql/udf.py --- @@ -157,7 +157,17 @@ def _create_judf(self): spark = SparkSession.builder.getOrCreate()

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 **[Test build #91263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91263/testReport)** for PR 20697 at commit

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3673/

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191481925 --- Diff: python/pyspark/sql/udf.py --- @@ -157,7 +157,17 @@ def _create_judf(self): spark = SparkSession.builder.getOrCreate()

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191481196 --- Diff: python/pyspark/sql/udf.py --- @@ -157,7 +157,17 @@ def _create_judf(self): spark = SparkSession.builder.getOrCreate()

[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21450 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91261/ Test FAILed. ---

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 **[Test build #91261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91261/testReport)** for PR 20697 at commit

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3672/

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...

2018-05-29 Thread gaborgsomogyi
GitHub user gaborgsomogyi opened a pull request: https://github.com/apache/spark/pull/21450 [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution where no main class is required. ## What changes were proposed in this pull request? With [PR

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 **[Test build #91261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91261/testReport)** for PR 20697 at commit

[GitHub] spark issue #21346: [SPARK-6237][NETWORK] Network-layer changes to allow str...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21346 **[Test build #4190 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4190/testReport)** for PR 21346 at commit

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21383 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21383 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91257/ Test PASSed. ---

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21383 **[Test build #91257 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91257/testReport)** for PR 21383 at commit

[GitHub] spark issue #21440: [SPARK-24307][CORE] Support reading remote cached partit...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21440 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3671/

[GitHub] spark issue #21440: [SPARK-24307][CORE] Support reading remote cached partit...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21440 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21440: [SPARK-24307][CORE] Support reading remote cached partit...

2018-05-29 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21440 thanks for the reviews @markhamstra @Ngone51 , I've updated the pr --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-05-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r191476566 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -659,6 +659,11 @@ private[spark] class BlockManager( * Get block

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 **[Test build #91260 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91260/testReport)** for PR 20697 at commit

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91260/ Test FAILed. ---

[GitHub] spark issue #21440: [SPARK-24307][CORE] Support reading remote cached partit...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21440 **[Test build #91259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91259/testReport)** for PR 21440 at commit

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 **[Test build #91260 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91260/testReport)** for PR 20697 at commit

[GitHub] spark issue #21442: [SPARK-24402] [SQL] Optimize `In` expression when only o...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21442 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3670/

[GitHub] spark issue #21442: [SPARK-24402] [SQL] Optimize `In` expression when only o...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21442 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20697: [SPARK-23010][k8s] Initial checkin of k8s integra...

2018-05-29 Thread ssuchter
Github user ssuchter commented on a diff in the pull request: https://github.com/apache/spark/pull/20697#discussion_r191474516 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/Logging.scala --- @@ -0,0 +1,35 @@

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread e-dorigatti
Github user e-dorigatti commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191473968 --- Diff: python/pyspark/sql/tests.py --- @@ -900,6 +900,22 @@ def __call__(self, x): self.assertEqual(f, f_.func)

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-05-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r191472949 --- Diff: core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala --- @@ -0,0 +1,154 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread e-dorigatti
Github user e-dorigatti commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191472884 --- Diff: python/pyspark/sql/udf.py --- @@ -157,7 +157,17 @@ def _create_judf(self): spark = SparkSession.builder.getOrCreate()

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-05-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r191472540 --- Diff: core/src/test/scala/org/apache/spark/io/ChunkedByteBufferFileRegionSuite.scala --- @@ -0,0 +1,154 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #21442: [SPARK-24402] [SQL] Optimize `In` expression when only o...

2018-05-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21442 LGTM, pending jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21442: [SPARK-24402] [SQL] Optimize `In` expression when...

2018-05-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21442#discussion_r191471982 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -219,10 +219,15 @@ object

[GitHub] spark issue #21442: [SPARK-24402] [SQL] Optimize `In` expression when only o...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21442 **[Test build #91258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91258/testReport)** for PR 21442 at commit

[GitHub] spark issue #21442: [SPARK-24402] [SQL] Optimize `In` expression when only o...

2018-05-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21442 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-05-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r191471289 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBufferFileRegion.scala --- @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191470695 --- Diff: python/pyspark/sql/tests.py --- @@ -900,6 +900,22 @@ def __call__(self, x): self.assertEqual(f, f_.func)

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191469832 --- Diff: python/pyspark/sql/tests.py --- @@ -900,6 +900,22 @@ def __call__(self, x): self.assertEqual(f, f_.func)

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191469562 --- Diff: python/pyspark/sql/udf.py --- @@ -157,7 +157,17 @@ def _create_judf(self): spark = SparkSession.builder.getOrCreate()

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21383 still lgtm if the tests pass --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21383 **[Test build #91257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91257/testReport)** for PR 21383 at commit

[GitHub] spark issue #21445: [SPARK-24404][SS] Increase currentEpoch when meet a Epoc...

2018-05-29 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/21445 Sure, we need to support this, but the approach in this PR doesn't work if it breaks existing tests. I think the best way to do it is to make the shuffle writer responsible for

[GitHub] spark issue #21449: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-05-29 Thread daniel-shields
Github user daniel-shields commented on the issue: https://github.com/apache/spark/pull/21449 I'm not sure that this behavior should be applied to all binary comparisons. It could result in unexpected behavior in some rare cases. For example: `df1.join(df2, df2['x'] < df1['x'])`

[GitHub] spark issue #21445: [SPARK-24404][SS] Increase currentEpoch when meet a Epoc...

2018-05-29 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21445 @LiangchangZ > In the real CP situation, reader and writer may be always in different tasks, right? Continuous mode already supports some valid use cases, and putting all in

[GitHub] spark pull request #21378: [SPARK-24326][Mesos] add support for local:// sch...

2018-05-29 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21378#discussion_r191449756 --- Diff: docs/running-on-mesos.md --- @@ -753,6 +753,16 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark pull request #21378: [SPARK-24326][Mesos] add support for local:// sch...

2018-05-29 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21378#discussion_r191449547 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -418,17 +417,33 @@

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-29 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21317 @mccheah gentle ping for merge. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91249/ Test PASSed. ---

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21426 **[Test build #91249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91249/testReport)** for PR 21426 at commit

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191441634 --- Diff: python/pyspark/sql/udf.py --- @@ -157,7 +157,17 @@ def _create_judf(self): spark = SparkSession.builder.getOrCreate()

[GitHub] spark pull request #21432: [SPARK-24373][SQL] Add AnalysisBarrier to Relatio...

2018-05-29 Thread WenboZhao
Github user WenboZhao commented on a diff in the pull request: https://github.com/apache/spark/pull/21432#discussion_r191441199 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -63,17 +63,17 @@ class RelationalGroupedDataset

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191437141 --- Diff: python/pyspark/sql/udf.py --- @@ -157,7 +157,17 @@ def _create_judf(self): spark = SparkSession.builder.getOrCreate()

[GitHub] spark issue #21405: [SPARK-24361][SQL] Polish code block manipulation API

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21405 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91251/ Test PASSed. ---

[GitHub] spark issue #21405: [SPARK-24361][SQL] Polish code block manipulation API

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21405 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20701 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3669/

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20701 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21405: [SPARK-24361][SQL] Polish code block manipulation API

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21405 **[Test build #91251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91251/testReport)** for PR 21405 at commit

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21260 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3539/ ---

[GitHub] spark issue #21448: [SPARK-24408][SQL][DOC] Move abs, bitwiseNOT, isnan, nan...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21448 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91250/ Test PASSed. ---

[GitHub] spark issue #21448: [SPARK-24408][SQL][DOC] Move abs, bitwiseNOT, isnan, nan...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21448 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21448: [SPARK-24408][SQL][DOC] Move abs, bitwiseNOT, isnan, nan...

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21448 **[Test build #91250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91250/testReport)** for PR 21448 at commit

[GitHub] spark issue #21360: [SPARK-24288] Enable preventing predicate pushdown

2018-05-29 Thread TomaszGaweda
Github user TomaszGaweda commented on the issue: https://github.com/apache/spark/pull/21360 Hi @maryannxue, thanks for the PR! Could you please rebase it? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21260 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3539/ ---

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21260 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3668/

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21260 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20701 **[Test build #91256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91256/testReport)** for PR 20701 at commit

[GitHub] spark pull request #21432: [SPARK-24373][SQL] Add AnalysisBarrier to Relatio...

2018-05-29 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21432#discussion_r191429427 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -63,17 +63,17 @@ class RelationalGroupedDataset

[GitHub] spark pull request #21432: [SPARK-24373][SQL] Add AnalysisBarrier to Relatio...

2018-05-29 Thread WenboZhao
Github user WenboZhao commented on a diff in the pull request: https://github.com/apache/spark/pull/21432#discussion_r191428684 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -63,17 +63,17 @@ class RelationalGroupedDataset

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread e-dorigatti
Github user e-dorigatti commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191427825 --- Diff: python/pyspark/sql/udf.py --- @@ -157,7 +157,17 @@ def _create_judf(self): spark = SparkSession.builder.getOrCreate()

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21260 **[Test build #91255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91255/testReport)** for PR 21260 at commit

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21260 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91255/ Test FAILed. ---

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21260 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-05-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21260 **[Test build #91255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91255/testReport)** for PR 21260 at commit

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191422043 --- Diff: python/pyspark/sql/udf.py --- @@ -157,7 +157,17 @@ def _create_judf(self): spark = SparkSession.builder.getOrCreate()

[GitHub] spark issue #21422: [Spark-24376][doc]Summary:compiling spark with scala-2.1...

2018-05-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21422 There was a similar(?) argument before - https://github.com/apache/spark/pull/18873#issuecomment-320864941, FYI. @gentlewangyu, let's just close this then. The gain is small anyway.

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-29 Thread e-dorigatti
Github user e-dorigatti commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r191418719 --- Diff: python/pyspark/sql/udf.py --- @@ -157,7 +157,17 @@ def _create_judf(self): spark = SparkSession.builder.getOrCreate()

[GitHub] spark issue #21422: [Spark-24376][doc]Summary:compiling spark with scala-2.1...

2018-05-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21422 Oops, I meant branch-2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21422: [Spark-24376][doc]Summary:compiling spark with scala-2.1...

2018-05-29 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21422 It is not valid for branch-2.3, 2.3 already removed the support of Scala 2.10. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21422: [Spark-24376][doc]Summary:compiling spark with scala-2.1...

2018-05-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21422 I don't know but the fix is open anyway and I thought we better just merge it since it's a valid fix to branch-2.3 anyway and there's no downside of it for example like making the backport

[GitHub] spark issue #21082: [SPARK-22239][SQL][Python] Enable grouped aggregate pand...

2018-05-29 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/21082 Hey @gatorsmile and @cloud-fan sorry to bother you but this I have addressed all comments on the PR and there are no new comments for a while. Could you please take a look and let me

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21397 Thank you for fixing this :-) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21397: [SPARK-24334] Fix race condition in ArrowPythonRunner ca...

2018-05-29 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/21397 Thanks all for review! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isInCollection in DataFrame AP...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3667/

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isInCollection in DataFrame AP...

2018-05-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21045: [SPARK-23931][SQL] Adds zip function to sparksql

2018-05-29 Thread DylanGuedes
Github user DylanGuedes commented on a diff in the pull request: https://github.com/apache/spark/pull/21045#discussion_r191412802 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -127,6 +127,165 @@ case class

[GitHub] spark pull request #21409: [SPARK-24365][SQL] Add Data Source write benchmar...

2018-05-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21409#discussion_r191412739 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceWriteBenchmark.scala --- @@ -0,0 +1,145 @@ +/* + * Licensed

[GitHub] spark pull request #21409: [SPARK-24365][SQL] Add Data Source write benchmar...

2018-05-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21409#discussion_r191412111 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceWriteBenchmark.scala --- @@ -0,0 +1,145 @@ +/* + * Licensed

[GitHub] spark pull request #21409: [SPARK-24365][SQL] Add Data Source write benchmar...

2018-05-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21409#discussion_r191411987 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceWriteBenchmark.scala --- @@ -0,0 +1,145 @@ +/* + * Licensed

<    1   2   3   4   5   6   >