[GitHub] spark issue #21669: [SPARK-23257][K8S] Kerberos Support for Spark on K8S

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3803/ ---

[GitHub] spark pull request #22612: [SPARK-24958] Add executors' process tree total m...

2018-10-08 Thread rezasafi
Github user rezasafi commented on a diff in the pull request: https://github.com/apache/spark/pull/22612#discussion_r223486041 --- Diff: core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #22614: [SPARK-25561][SQL] Implement a new config to control par...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22614 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97119/ Test PASSed. ---

[GitHub] spark issue #22614: [SPARK-25561][SQL] Implement a new config to control par...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22614 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22614: [SPARK-25561][SQL] Implement a new config to control par...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22614 **[Test build #97119 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97119/testReport)** for PR 22614 at commit

[GitHub] spark issue #22627: [SPARK-25639] [DOCS] Added docs for foreachBatch, python...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22627 **[Test build #97129 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97129/testReport)** for PR 22627 at commit

[GitHub] spark issue #21669: [SPARK-23257][K8S] Kerberos Support for Spark on K8S

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 **[Test build #97128 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97128/testReport)** for PR 21669 at commit

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-10-08 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r223483718 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/HadoopConfExecutorFeatureStep.scala --- @@ -0,0 +1,51

[GitHub] spark pull request #22612: [SPARK-24958] Add executors' process tree total m...

2018-10-08 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/22612#discussion_r223467439 --- Diff: core/src/main/scala/org/apache/spark/executor/ProcfsBasedSystems.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #22667: [SPARK-25673][BUILD] Remove Travis CI which enables Java...

2018-10-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22667 Thank you for taking care of this, @HyukjinKwon . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #22643: [SPARK-25630][TEST] Reduce test time of HadoopFsR...

2018-10-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22643 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-10-08 Thread michal-databricks
Github user michal-databricks commented on the issue: https://github.com/apache/spark/pull/21688 @pgandhi999 thanks for the changes. I will review them tomorrow. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21809: [SPARK-24851][UI] Map a Stage ID to it's Associated Job ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21809 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97120/ Test FAILed. ---

[GitHub] spark issue #21809: [SPARK-24851][UI] Map a Stage ID to it's Associated Job ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21809 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21809: [SPARK-24851][UI] Map a Stage ID to it's Associated Job ...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21809 **[Test build #97120 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97120/testReport)** for PR 21809 at commit

[GitHub] spark issue #22673: [SPARK-20144] Allow reading files in order with spark.sq...

2018-10-08 Thread darabos
Github user darabos commented on the issue: https://github.com/apache/spark/pull/22673 > @darabos . Thank you for trying to make a contribution. However, we had better discuss on that JIRA first before making a PR. Especially, for SPARK-20144 which is discussed already, it does.

[GitHub] spark issue #22643: [SPARK-25630][TEST] Reduce test time of HadoopFsRelation...

2018-10-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22643 LGTM Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-10-08 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r223481427 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/KerberosConfDriverFeatureStep.scala --- @@ -0,0 +1,161

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-10-08 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r223480986 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/HadoopConfExecutorFeatureStep.scala --- @@ -0,0 +1,51

[GitHub] spark pull request #22627: [SPARK-25639] [DOCS] Added docs for foreachBatch,...

2018-10-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/22627#discussion_r223481016 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1989,22 +2026,211 @@ head(sql("select * from aggregates")) -#

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-10-08 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r223480732 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/KerberosConfDriverFeatureStep.scala --- @@ -0,0 +1,161

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97121/ Test FAILed. ---

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21688 **[Test build #97121 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97121/testReport)** for PR 21688 at commit

[GitHub] spark pull request #22627: [SPARK-25639] [DOCS] Added docs for foreachBatch,...

2018-10-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/22627#discussion_r223479786 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -2709,6 +2935,78 @@ write.stream(aggDF, "memory", outputMode = "complete", checkpointLocation

[GitHub] spark pull request #22627: [SPARK-25639] [DOCS] Added docs for foreachBatch,...

2018-10-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/22627#discussion_r223479414 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1989,22 +2026,211 @@ head(sql("select * from aggregates")) -# Using

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223478828 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -452,46 +452,68 @@ case class RangeExec(range:

[GitHub] spark issue #22623: [SPARK-25636][CORE] spark-submit cuts off the failure re...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22623 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97127/ Test FAILed. ---

[GitHub] spark issue #22623: [SPARK-25636][CORE] spark-submit cuts off the failure re...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22623 **[Test build #97127 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97127/testReport)** for PR 22623 at commit

[GitHub] spark issue #22623: [SPARK-25636][CORE] spark-submit cuts off the failure re...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22623 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22664: [SPARK-25662][TEST] Refactor DataSourceReadBenchm...

2018-10-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22664#discussion_r223476989 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala --- @@ -34,10 +34,15 @@ import

[GitHub] spark issue #22623: [SPARK-25636][CORE] spark-submit cuts off the failure re...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22623 **[Test build #97127 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97127/testReport)** for PR 22623 at commit

[GitHub] spark pull request #22653: [SPARK-25659][PYTHON][TEST] Test type inference s...

2018-10-08 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/22653#discussion_r223475373 --- Diff: python/pyspark/sql/tests.py --- @@ -1149,6 +1149,75 @@ def test_infer_schema(self): result = self.spark.sql("SELECT l[0].a from

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21710 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97125/ Test PASSed. ---

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21710 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22624: [SPARK-23781][CORE] Add base class for token rene...

2018-10-08 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/22624#discussion_r223472701 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/AbstractCredentialRenewer.scala --- @@ -0,0 +1,224 @@ +/* + * Licensed to the

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21710 **[Test build #97125 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97125/testReport)** for PR 21710 at commit

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223474906 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -345,6 +345,61 @@ trait CodegenSupport extends SparkPlan

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223474853 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -452,46 +452,73 @@ case class RangeExec(range:

[GitHub] spark issue #22614: [SPARK-25561][SQL] Implement a new config to control par...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22614 **[Test build #97126 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97126/testReport)** for PR 22614 at commit

[GitHub] spark issue #22615: [SPARK-25016][BUILD][CORE] Remove support for Hadoop 2.6

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22615 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97116/ Test PASSed. ---

[GitHub] spark issue #22615: [SPARK-25016][BUILD][CORE] Remove support for Hadoop 2.6

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22615 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22614: [SPARK-25561][SQL] Implement a new config to cont...

2018-10-08 Thread kmanamcheri
Github user kmanamcheri commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r223473324 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,45 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #22615: [SPARK-25016][BUILD][CORE] Remove support for Hadoop 2.6

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22615 **[Test build #97116 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97116/testReport)** for PR 22615 at commit

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22466 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22466 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97118/ Test PASSed. ---

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22466 **[Test build #97118 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97118/testReport)** for PR 22466 at commit

[GitHub] spark pull request #22614: [SPARK-25561][SQL] Implement a new config to cont...

2018-10-08 Thread kmanamcheri
Github user kmanamcheri commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r223469446 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientSuite.scala --- @@ -79,12 +82,30 @@ class HiveClientSuite(version:

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223468018 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -518,56 +521,81 @@ class SQLMetricsSuite extends

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-10-08 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r223467868 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/KerberosConfDriverFeatureStep.scala --- @@ -0,0 +1,161

[GitHub] spark pull request #22612: [SPARK-24958] Add executors' process tree total m...

2018-10-08 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/22612#discussion_r223466260 --- Diff: core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala --- @@ -59,6 +60,43 @@ case object JVMOffHeapMemory extends

[GitHub] spark issue #21755: Doc fix: The Imputer is an Estimator

2018-10-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21755 Minor stuff: we usually tag this with `[MINOR]` in the title to be clear there's no JIRA. Also ideal to batch together small related changes but I don't know that there was anything else to do here.

[GitHub] spark issue #21755: Doc fix: The Imputer is an Estimator

2018-10-08 Thread zoltanctoth
Github user zoltanctoth commented on the issue: https://github.com/apache/spark/pull/21755 @srowen Just about to submit a new doc relates pull request. Wondering if your `PS see https://spark.apache.org/contributing.html` line referred to anything specific about how I should issue

[GitHub] spark pull request #22614: [SPARK-25561][SQL] Implement a new config to cont...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r223462348 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientSuite.scala --- @@ -79,12 +82,30 @@ class HiveClientSuite(version: String)

[GitHub] spark pull request #22614: [SPARK-25561][SQL] Implement a new config to cont...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r223461273 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,45 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request #22614: [SPARK-25561][SQL] Implement a new config to cont...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r223459567 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -544,6 +544,15 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-10-08 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r223461972 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/HadoopConfExecutorFeatureStep.scala --- @@ -0,0 +1,51

[GitHub] spark issue #22624: [SPARK-23781][CORE] Add base class for token renewal fun...

2018-10-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22624 Given the lack of reviews let me ping some people: @skonto @jerryshao --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-10-08 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r223461492 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/KerberosConfDriverFeatureStep.scala --- @@ -0,0 +1,161

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97124/ Test PASSed. ---

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #97124 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97124/testReport)** for PR 22295 at commit

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-10-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r223460909 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3965,6 +4034,248 @@ object

[GitHub] spark issue #22635: [SPARK-25591][PySpark][SQL] Avoid overwriting deserializ...

2018-10-08 Thread AbdealiJK
Github user AbdealiJK commented on the issue: https://github.com/apache/spark/pull/22635 @cloud-fan @viirya Any chance of this making it into 2.4 ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-10-08 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r223459186 --- Diff: docs/sql-programming-guide.md --- @@ -1890,6 +1890,10 @@ working with timestamps in `pandas_udf`s to get the best performance, see #

[GitHub] spark issue #22645: [SPARK-25566][SPARK-25567][WEBUI][SQL]Support pagination...

2018-10-08 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/22645 Hi @srowen , There is one behavior change this PR introduces, which is correct. Sorting Job Ids in the previous versions of spark was not proper. After the PR the sorting is proper.

[GitHub] spark pull request #22628: [SPARK-25641] Change the spark.shuffle.server.chu...

2018-10-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22628 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22627: [SPARK-25639] [DOCS] Added docs for foreachBatch, python...

2018-10-08 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/22627 @holdenk yeah, i intend to backport this to 2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22627: [SPARK-25639] [DOCS] Added docs for foreachBatch,...

2018-10-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/22627#discussion_r223456294 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -2709,6 +2935,78 @@ write.stream(aggDF, "memory", outputMode = "complete", checkpointLocation

[GitHub] spark pull request #22627: [SPARK-25639] [DOCS] Added docs for foreachBatch,...

2018-10-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/22627#discussion_r223456079 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1989,22 +2026,211 @@ head(sql("select * from aggregates")) -# Using

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97115/ Test PASSed. ---

[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...

2018-10-08 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22628 pulled into master, thanks @redsanket --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #97115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97115/testReport)** for PR 22598 at commit

[GitHub] spark pull request #22623: [SPARK-25636][CORE] spark-submit cuts off the fai...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22623#discussion_r223454115 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -74,20 +74,26 @@ trait TestPrematureExit { @volatile var

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21710 **[Test build #97125 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97125/testReport)** for PR 21710 at commit

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21710 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22623: [SPARK-25636][CORE] spark-submit cuts off the fai...

2018-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22623#discussion_r223453846 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -74,20 +74,26 @@ trait TestPrematureExit { @volatile var

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21710 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22623: [SPARK-25636][CORE] spark-submit cuts off the fai...

2018-10-08 Thread devaraj-kavali
Github user devaraj-kavali commented on a diff in the pull request: https://github.com/apache/spark/pull/22623#discussion_r223453234 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -74,20 +74,26 @@ trait TestPrematureExit { @volatile

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97114/ Test PASSed. ---

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #97114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97114/testReport)** for PR 22598 at commit

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22673: [SPARK-20144] Allow reading files in order with spark.sq...

2018-10-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22673 @darabos . Thank you for trying to make a contribution. However, we had better discuss on that JIRA first before making a PR. Especially, for SPARK-20144 which is discussed already, it does.

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r223451952 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -597,6 +597,17 @@ package object config { .checkValue(v =>

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222785870 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -117,6 +117,12 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222783407 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -597,6 +597,17 @@ package object config { .checkValue(v =>

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r223421506 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -503,6 +505,92 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222813512 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -597,6 +597,17 @@ package object config { .checkValue(v =>

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222814235 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222811527 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222810125 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r223421728 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -503,6 +505,92 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222785940 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -117,6 +117,12 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222811152 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222816923 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -414,9 +425,48 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #97124 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97124/testReport)** for PR 22295 at commit

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r222810004 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +421,63 @@ private[spark] class TaskSchedulerImpl(

<    1   2   3   4   5   6   >