[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3575/ ---

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3708/

[GitHub] spark pull request #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files...

2018-05-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21426 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91310/ Test FAILed. ---

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 **[Test build #91310 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91310/testReport)** for PR 20697 at commit

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20697 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files...

2018-05-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21426#discussion_r191855502 --- Diff: core/src/main/scala/org/apache/spark/deploy/PythonRunner.scala --- @@ -153,4 +154,30 @@ object PythonRunner { .map { p => formatPath(p,

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-30 Thread ssuchter
Github user ssuchter commented on the issue: https://github.com/apache/spark/pull/20697 Any more issues that should be addressed now? Is this ready for merge? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20697 **[Test build #91310 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91310/testReport)** for PR 20697 at commit

[GitHub] spark pull request #20697: [SPARK-23010][k8s] Initial checkin of k8s integra...

2018-05-30 Thread ssuchter
Github user ssuchter commented on a diff in the pull request: https://github.com/apache/spark/pull/20697#discussion_r191855713 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend/IntegrationTestBackend.scala ---

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-05-30 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21459 What's driving this (is it java 9)? I'm in general scared by core library updates like this. Maybe Spark 3.0 is a good time (and we should just do it this year). ---

[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...

2018-05-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21450#discussion_r191851612 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java --- @@ -229,7 +238,7 @@ args.add(join(",", pyFiles));

[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...

2018-05-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21450#discussion_r191851844 --- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java --- @@ -344,8 +365,15 @@ private SparkSubmitCommandBuilder

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21331 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3707/

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21331 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21454: [SPARK-24337][Core] Improve error messages for Sp...

2018-05-30 Thread PenguinToast
Github user PenguinToast commented on a diff in the pull request: https://github.com/apache/spark/pull/21454#discussion_r191849201 --- Diff: core/src/test/scala/org/apache/spark/SparkConfSuite.scala --- @@ -339,6 +341,38 @@ class SparkConfSuite extends SparkFunSuite with

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21331 **[Test build #91309 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91309/testReport)** for PR 21331 at commit

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-05-30 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21459 @srowen From the release note, http://asm.ow2.io/versions.html , the differences between 5.2 to 6.0 are > Codebase migrated to gitlab (feature requests 317617, 317619, 317542) Support

[GitHub] spark issue #21458: [SPARK-24419] [Build] Upgrade SBT to 0.13.17 with Scala ...

2018-05-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21458 Thank you for working on this, @dbtsai ! :D --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to suppor...

2018-05-30 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21459#discussion_r191846953 --- Diff: pom.xml --- @@ -313,13 +313,13 @@ chill-java ${chill.version} -

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21331 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21331 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21463: [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising Stop...

2018-05-30 Thread e-dorigatti
Github user e-dorigatti closed the pull request at: https://github.com/apache/spark/pull/21463 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21463: [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising StopIterati...

2018-05-30 Thread e-dorigatti
Github user e-dorigatti commented on the issue: https://github.com/apache/spark/pull/21463 @HyukjinKwon go ahead, no problem --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the external c...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20795 ping @kevinyu98 @dilipbiswal --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #91308 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91308/testReport)** for PR 19691 at commit

[GitHub] spark pull request #21378: [SPARK-24326][Mesos] add support for local:// sch...

2018-05-30 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21378#discussion_r191842162 --- Diff: docs/running-on-mesos.md --- @@ -753,6 +753,16 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark issue #21463: [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising StopIterati...

2018-05-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21463 @e-dorigatti, I think this should really be cleaned up in the master branch. Mind if I ask to give a try with fixing it in worker side and revert 0ebb0c0d4dd3e192464dc5e0e6f01efa55b945ed in the

[GitHub] spark issue #21463: [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising StopIterati...

2018-05-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21463 Merged to branch-2.3. @e-dorigatti, in case of other branches, it should be manually closed by you. --- - To

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-05-30 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13599 One point that I'm reminded of looking at this is the lack of tests for important functionality is probably not an acceptable solution. I understand testing this in Jenkins could be complicated but

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-05-30 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21068 well the downside to that and not just failing the application is similar to what @squito was mentioning, if the cluster is just busy and you can't get containers on those last few nodes, it

[GitHub] spark pull request #21443: [SPARK-24369][SQL] Correct handling for multiple ...

2018-05-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21443 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21449: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-05-30 Thread daniel-shields
Github user daniel-shields commented on the issue: https://github.com/apache/spark/pull/21449 @mgaido91 I looked at the test failures and I think the changes to the Dataset,resolve method are causing havoc. Consider the Dataset.drop method with the following signature: ` def

[GitHub] spark issue #21443: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-05-30 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21443 good catch! LGTM, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21247: [SPARK-24190] Separating JSONOptions for read

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21247#discussion_r191831524 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2313,25 +2314,6 @@ class JsonSuite extends

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21439 cc @gengliangwang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r191827211 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala --- @@ -110,14 +114,81 @@ abstract class

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r191820709 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala --- @@ -156,6 +156,12 @@ class CSVOptions( val

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r191824714 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala --- @@ -110,14 +114,81 @@ abstract class

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r191819710 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -130,14 +130,16 @@ class CSVFileFormat

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r191819577 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala --- @@ -206,12 +290,17 @@ object

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r191826405 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -499,6 +503,11 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r191026857 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala --- @@ -110,14 +114,81 @@ abstract class

[GitHub] spark pull request #21428: [SPARK-24235][SS] Implement continuous shuffle wr...

2018-05-30 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/21428#discussion_r191823576 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleWriter.scala --- @@ -0,0 +1,54 @@

[GitHub] spark pull request #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP P...

2018-05-30 Thread DazhuangSu
Github user DazhuangSu commented on a diff in the pull request: https://github.com/apache/spark/pull/19691#discussion_r191823358 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -293,6 +313,15 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #21463: [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising StopIterati...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21463 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21463: [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising StopIterati...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21463 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91306/ Test PASSed. ---

[GitHub] spark issue #21463: [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising StopIterati...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21463 **[Test build #91306 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91306/testReport)** for PR 21463 at commit

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-05-30 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21439 Thank you @maropu for your review of the PR. > better to add tests in json-functions.sql? What kind of tests would you expect in `json-functions.sql`. Probably you would expect

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21068 **[Test build #91307 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91307/testReport)** for PR 21068 at commit

[GitHub] spark issue #21403: [SPARK-24341][WIP][SQL] Support IN subqueries with struc...

2018-05-30 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21403 I am encountering big issues in enforcing the behavior we mentioned. The problem is that we cannot really distinguish the cases: - `... (a, b) in (select ...)` - `... from (select (a, b)

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-30 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/21231 ping @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91302/ Test PASSed. ---

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #91302 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91302/testReport)** for PR 13599 at commit

[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers

2018-05-30 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 @gatorsmile @cloud-fan @HyukjinKwon @gengliangwang Could you look at the PR one more time. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP P...

2018-05-30 Thread DazhuangSu
Github user DazhuangSu commented on a diff in the pull request: https://github.com/apache/spark/pull/19691#discussion_r191805297 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -515,28 +515,66 @@ case class

[GitHub] spark issue #21461: [SPARK-23754][Python] Backport bugfix

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21461 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91301/ Test FAILed. ---

[GitHub] spark issue #21461: [SPARK-23754][Python] Backport bugfix

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21461 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21461: [SPARK-23754][Python] Backport bugfix

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21461 **[Test build #91301 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91301/testReport)** for PR 21461 at commit

[GitHub] spark issue #21463: [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising StopIterati...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21463 **[Test build #91306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91306/testReport)** for PR 21463 at commit

[GitHub] spark pull request #21082: [SPARK-22239][SQL][Python] Enable grouped aggrega...

2018-05-30 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21082#discussion_r191801865 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/WindowInPandasExec.scala --- @@ -0,0 +1,173 @@ +/* + * Licensed to the

[GitHub] spark pull request #21461: [SPARK-23754][Python] Backport bugfix

2018-05-30 Thread e-dorigatti
Github user e-dorigatti closed the pull request at: https://github.com/apache/spark/pull/21461 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21463: [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising StopIterati...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21463 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21463: [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising Stop...

2018-05-30 Thread e-dorigatti
GitHub user e-dorigatti opened a pull request: https://github.com/apache/spark/pull/21463 [SPARK-23754][BRANCH-2.3][PYTHON] Re-raising StopIteration in client code ## What changes are proposed Make sure that `StopIteration`s raised in users' code do not silently interrupt

[GitHub] spark pull request #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP P...

2018-05-30 Thread DazhuangSu
Github user DazhuangSu commented on a diff in the pull request: https://github.com/apache/spark/pull/19691#discussion_r191795550 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -515,28 +515,66 @@ case class

[GitHub] spark issue #21346: [SPARK-6237][NETWORK] Network-layer changes to allow str...

2018-05-30 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21346 yeah I see what you're saying about better error handling, but I'd really rather not take that on here. I think some prior attempts at solving the 2gb limit have tried to take on too much, and I'd

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-05-30 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/21068 @tgravescs what about removing YARN_BLACKLIST_MAX_NODE_BLACKLIST_RATIO config and when the set of backlisted nodes reaches numClusterNodes I stop synchronising the backlisted nodes toward YARN

[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21450 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91299/ Test PASSed. ---

[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21450 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21450 **[Test build #91299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91299/testReport)** for PR 21450 at commit

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21383 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21383 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91300/ Test FAILed. ---

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21383 **[Test build #91300 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91300/testReport)** for PR 21383 at commit

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21462 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3573/ ---

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21462 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91305/ Test PASSed. ---

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21462 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3573/ ---

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21462 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21462 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21462 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3706/

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21462 **[Test build #91305 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91305/testReport)** for PR 21462 at commit

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21462 **[Test build #91305 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91305/testReport)** for PR 21462 at commit

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r191775677 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh --- @@ -46,12 +46,6 @@ shift 1

[GitHub] spark issue #21449: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21449 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21449: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21449 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91303/ Test FAILed. ---

[GitHub] spark issue #21449: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21449 **[Test build #91303 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91303/testReport)** for PR 21449 at commit

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread skonto
GitHub user skonto opened a pull request: https://github.com/apache/spark/pull/21462 [SPARK-24428][K8S] Fix unused code ## What changes were proposed in this pull request? Remove code that is misleading and is a leftover from previous implementation. ## How was

[GitHub] spark issue #21448: [SPARK-24408][SQL][DOC] Move abs, bitwiseNOT, isnan, nan...

2018-05-30 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21448 `abs` does seem to belong with math functions. `isnan` and `nanvl` are sort of math related but to me feel more like `isnull`. It's more about testing representations of a value, not math

[GitHub] spark pull request #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to suppor...

2018-05-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21459#discussion_r191768841 --- Diff: pom.xml --- @@ -313,13 +313,13 @@ chill-java ${chill.version} -

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-30 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r191762829 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala --- @@ -328,4 +328,19 @@ package object config {

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91297/ Test PASSed. ---

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21370 **[Test build #91297 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91297/testReport)** for PR 21370 at commit

[GitHub] spark pull request #21428: [SPARK-24235][SS] Implement continuous shuffle wr...

2018-05-30 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21428#discussion_r191747894 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleReader.scala --- @@ -48,7 +46,7 @@

[GitHub] spark issue #21261: [SPARK-24203][core] Add spark.executor.bindAddress

2018-05-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21261 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-05-30 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21459 cc @srowen and @rxin for more eyes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21459: [SPARK-24420][Build] Upgrade ASM to 6.1 to support JDK9+

2018-05-30 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21459 @HyukjinKwon Once Spark can be built against JDK9+, we'll need to figure out how we want to set it up in Jenkins. We can do two builds, and one for JDK8 and another one for JDK9+ for each PR, but

[GitHub] spark pull request #21458: [SPARK-24419] [Build] Upgrade SBT to 0.13.17 with...

2018-05-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21458 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21458: [SPARK-24419] [Build] Upgrade SBT to 0.13.17 with Scala ...

2018-05-30 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21458 Merged into master. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

<    1   2   3   4   5   >