[GitHub] spark pull request #22090: [DOCS] Fixed NDCG formula issues

2018-08-16 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22090#discussion_r210772634 --- Diff: docs/mllib-evaluation-metrics.md --- @@ -461,11 +461,11 @@ $$rel_D(r) = \begin{cases}1 & \text{if $r \in D$}, \\ 0 & \text{otherwise}.\end{

[GitHub] spark pull request #22090: [DOCS] Fixed NDCG formula issues

2018-08-16 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22090#discussion_r210772686 --- Diff: docs/mllib-evaluation-metrics.md --- @@ -461,11 +461,11 @@ $$rel_D(r) = \begin{cases}1 & \text{if $r \in D$}, \\ 0 & \text{otherwise}.\end{

[GitHub] spark issue #22125: [DOCS] Fix cloud-integration.md Typo

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22125 **[Test build #4279 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4279/testReport)** for PR 22125 at commit

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21584 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21584 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94862/ Test PASSed. ---

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21584 **[Test build #94862 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94862/testReport)** for PR 21584 at commit

[GitHub] spark issue #22125: [DOCS] Fix cloud-integration.md Typo

2018-08-16 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22125 OK, we should bundle these, but w/e --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22125: [DOCS] Fix cloud-integration.md Typo

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22125 **[Test build #4279 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4279/testReport)** for PR 22125 at commit

[GitHub] spark issue #20725: [SPARK-23555][PYTHON] Add BinaryType support for Arrow i...

2018-08-16 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/20725 still having problems getting this to pass: ``` [error] (sql-kafka-0-10/test:test) sbt.TestsFailedException: Tests unsuccessful [error] (core/test:test) sbt.TestsFailedException:

[GitHub] spark issue #22126: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of val...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22126 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94863/ Test PASSed. ---

[GitHub] spark issue #22126: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of val...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22126 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22126: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of val...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22126 **[Test build #94863 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94863/testReport)** for PR 22126 at commit

[GitHub] spark issue #22128: Add test_slice() to streaming BasicOperations

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22128 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22128: Add test_slice() to streaming BasicOperations

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22128 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22127: [SPARK-25032][SQL] fix drop database issue

2018-08-16 Thread bomeng
Github user bomeng commented on the issue: https://github.com/apache/spark/pull/22127 Good points. I will leave it open for any suggestions for improving the user experience.. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22128: Add test_slice() to streaming BasicOperations

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22128 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22128: Add test_slice() to streaming BasicOperations

2018-08-16 Thread cclauss
GitHub user cclauss opened a pull request: https://github.com/apache/spark/pull/22128 Add test_slice() to streaming BasicOperations As suggested in https://github.com/apache/spark/pull/20838#pullrequestreview-139118618 ## What changes were proposed in this pull request?

[GitHub] spark pull request #21909: [SPARK-24959][SQL] Speed up count() for JSON and ...

2018-08-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21909#discussion_r210767018 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2223,21 +2223,31 @@ class JsonSuite extends

[GitHub] spark pull request #22108: [SPARK-25092][SQL][FOLLOWUP] Add RewriteCorrelate...

2018-08-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22108 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21909: [SPARK-24959][SQL] Speed up count() for JSON and ...

2018-08-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21909#discussion_r210765672 --- Diff: docs/sql-programming-guide.md --- @@ -1894,6 +1894,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see - In

[GitHub] spark issue #22108: [SPARK-25092][SQL][FOLLOWUP] Add RewriteCorrelatedScalar...

2018-08-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22108 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22127: [SPARK-25032][SQL] fix drop database issue

2018-08-16 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22127 @bomeng There seems to be bit of history to this :-) . Please check https://github.com/apache/spark/pull/15011 where we decided against silently switching to "default" database. ---

[GitHub] spark pull request #22117: [SPARK-23654][BUILD] remove jets3t as a dependenc...

2018-08-16 Thread steveloughran
Github user steveloughran closed the pull request at: https://github.com/apache/spark/pull/22117 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22081: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/22081 Thanks. Two less JARs on the CP to keep up to date —what more can anyone want? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21950: [SPARK-24914][SQL][WIP] Add configuration to avoid OOM d...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21950 **[Test build #94869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94869/testReport)** for PR 21950 at commit

[GitHub] spark issue #21135: [SPARK-24060][TEST] StreamingSymmetricHashJoinHelperSuit...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21135 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21561: [SPARK-24555][ML] logNumExamples in KMeans/BiKM/G...

2018-08-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21561 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21561: [SPARK-24555][ML] logNumExamples in KMeans/BiKM/GMM/AFT/...

2018-08-16 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21561 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22112 Thanks for the clarification, but I guess my point is with your last statement: > - with assumption that we will expand solution to cover all later. If we document this and say

[GitHub] spark issue #21909: [SPARK-24959][SQL] Speed up count() for JSON and CSV

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21909 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94860/ Test PASSed. ---

[GitHub] spark issue #21909: [SPARK-24959][SQL] Speed up count() for JSON and CSV

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21909 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21909: [SPARK-24959][SQL] Speed up count() for JSON and CSV

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21909 **[Test build #94860 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94860/testReport)** for PR 21909 at commit

[GitHub] spark issue #22127: [SPARK-25032][SQL] fix drop database issue

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22127 **[Test build #94868 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94868/testReport)** for PR 22127 at commit

[GitHub] spark issue #22127: [SPARK-25032][SQL] fix drop database issue

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22127 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22127: [SPARK-25032][SQL] fix drop database issue

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22127 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22127: [SPARK-25032][SQL] fix drop database issue

2018-08-16 Thread bomeng
GitHub user bomeng opened a pull request: https://github.com/apache/spark/pull/22127 [SPARK-25032][SQL] fix drop database issue ## What changes were proposed in this pull request? When user tries to drop the current database (other than default database), after the database is

[GitHub] spark issue #22114: [SPARK-24938][Core] Prevent Netty from using onheap memo...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22114 **[Test build #94867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94867/testReport)** for PR 22114 at commit

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/22112 @tgravescs I was specifically in agreement with > Personally I don't want to talk about implementation until we decide what we want our semantics to be around the unordered operations because

[GitHub] spark issue #22114: [SPARK-24938][Core] Prevent Netty from using onheap memo...

2018-08-16 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22114 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22114: [SPARK-24938][Core] Prevent Netty from using onheap memo...

2018-08-16 Thread NiharS
Github user NiharS commented on the issue: https://github.com/apache/spark/pull/22114 Tried with a significantly larger input, both with and without the change. They ran in just about the same time. --- - To

[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94866/ Test FAILed. ---

[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94866 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94866/testReport)** for PR 21990 at commit

[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21990 **[Test build #94866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94866/testReport)** for PR 21990 at commit

[GitHub] spark issue #21950: [SPARK-24914][SQL][WIP] Add configuration to avoid OOM d...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21950 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94853/ Test PASSed. ---

[GitHub] spark issue #21950: [SPARK-24914][SQL][WIP] Add configuration to avoid OOM d...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21950 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21950: [SPARK-24914][SQL][WIP] Add configuration to avoid OOM d...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21950 **[Test build #94853 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94853/testReport)** for PR 21950 at commit

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22124 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22124 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94861/ Test FAILed. ---

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22124 **[Test build #94861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94861/testReport)** for PR 22124 at commit

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21221 **[Test build #94865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94865/testReport)** for PR 21221 at commit

[GitHub] spark issue #22107: [SPARK-25117][R] Add EXEPT ALL and INTERSECT ALL support...

2018-08-16 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22107 @felixcheung I have incorporated the comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #22126: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities...

2018-08-16 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/22126#discussion_r210724650 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala --- @@ -363,9 +363,9 @@ class

[GitHub] spark issue #22045: [SPARK-23940][SQL] Add transform_values SQL function

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22045 **[Test build #94864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94864/testReport)** for PR 22045 at commit

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22112 @mridulm so just to clarify are you agreeing that we need to decide on what we do with zip and others or are you agreeing that we should document these as unordered actions thus retries might be

[GitHub] spark pull request #21889: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-08-16 Thread ajacques
Github user ajacques closed the pull request at: https://github.com/apache/spark/pull/21889 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21584 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2256/ ---

[GitHub] spark issue #22126: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of val...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22126 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21584 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22126: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of val...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22126 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22126: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of val...

2018-08-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22126 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21584 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22081: [SPARK-23654][BUILD] remove jets3t as a dependenc...

2018-08-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22081 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-16 Thread ajacques
Github user ajacques commented on the issue: https://github.com/apache/spark/pull/21889 Thanks for the response all. @mailman If it's really your preference, I will create a PR against that branch and close this one. My intention was never to take away from your efforts, and I still

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-08-16 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/21584 Let's merge at the end of the day pacific time (~5PM-ish) on Friday, August 17, pending any additional feedback on the mailing list thread discussing the subject of including this in 2.4. ---

[GitHub] spark issue #22081: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22081 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21584 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2256/ ---

[GitHub] spark issue #22119: [WIP][SPARK-25129][SQL] Revert mapping com.databricks.sp...

2018-08-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22119 +1 for @tgravescs 's comments. In terms of usability, the mapping and configuration will be easier for the most customers. For the following @gengliangwang 's comment, technically

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-16 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/22112 I agree @tgravescs, I was looking at the implementation to understand what the expectations are wrt newly introduced methods/fields and whether they make sense : I did not see any details

[GitHub] spark issue #22123: [SPARK-25134][SQL] Csv column pruning with checking of h...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22123 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94854/ Test PASSed. ---

[GitHub] spark issue #22123: [SPARK-25134][SQL] Csv column pruning with checking of h...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22123 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22123: [SPARK-25134][SQL] Csv column pruning with checking of h...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22123 **[Test build #94854 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94854/testReport)** for PR 22123 at commit

[GitHub] spark issue #22126: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of val...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22126 **[Test build #94863 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94863/testReport)** for PR 22126 at commit

[GitHub] spark issue #22126: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of val...

2018-08-16 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22126 cc @mn-mikke --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22126: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities...

2018-08-16 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22126 [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of value arguments should be true. ## What changes were proposed in this pull request? This is a follow-up pr of #22017 which added

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21584 **[Test build #94862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94862/testReport)** for PR 21584 at commit

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-08-16 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/21584 This PR has been updated to pass Jenkins by removing the `with RTestsSuite` line in `KubernetesSuite`. As such, this feature may be merged and the `with RTestsSuite` will be re-included in a

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-16 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/21320 > I see no point of leaving this PR open. I don't agree with you on that point, and I've expressed my view in https://github.com/apache/spark/pull/21889#issuecomment-413655304. ---

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-16 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/21889 Essentially, this PR was created to take the management of #21320 out of my hands, with a view towards facilitating its incorporation into Spark 2.4. It was my suggestion, one based in frustration.

[GitHub] spark issue #22125: [DOCS] Fix cloud-integration.md Typo

2018-08-16 Thread KraFusion
Github user KraFusion commented on the issue: https://github.com/apache/spark/pull/22125 @kiszk PR created yesterday for ```configurations.md``` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22125: [DOCS] Fix cloud-integration.md Typo

2018-08-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22125 Thanks, would it possible to address similar issues? For example, in `configurations.md`. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21919: [SPARK-24933][SS] Report numOutputRows in SinkPro...

2018-08-16 Thread vackosar
Github user vackosar commented on a diff in the pull request: https://github.com/apache/spark/pull/21919#discussion_r210708107 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -254,3 +259,10 @@ class SinkProgress protected[sql]( }

[GitHub] spark issue #22125: [DOCS] Fix cloud-integration.md Typo

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22125 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22125: [DOCS] Fix cloud-integration.md Typo

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22125 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22125: [DOCS] Fix cloud-integration.md Typo

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22125 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21919: [SPARK-24933][SS] Report numOutputRows in SinkProgress

2018-08-16 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21919 LGTM overall except one minor comment. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21919: [SPARK-24933][SS] Report numOutputRows in SinkPro...

2018-08-16 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/21919#discussion_r210707152 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -254,3 +259,10 @@ class SinkProgress protected[sql](

[GitHub] spark pull request #22125: [DOCS] Fix cloud-integration.md Typo

2018-08-16 Thread KraFusion
GitHub user KraFusion opened a pull request: https://github.com/apache/spark/pull/22125 [DOCS] Fix cloud-integration.md Typo Corrected typo; changed spark-default.conf to spark-defaults.conf You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/22117 Test failure in ` org.apache.spark.sql.hive.client.HiveClientSuites.(It is not a test it is a sbt.testing.SuiteSelector)`: ``` Caused by: sbt.ForkMain$ForkError:

[GitHub] spark pull request #21909: [SPARK-24959][SQL] Speed up count() for JSON and ...

2018-08-16 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21909#discussion_r210704902 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2223,21 +2223,31 @@ class JsonSuite extends

[GitHub] spark issue #21819: [SPARK-24863][SS] Report Kafka offset lag as a custom me...

2018-08-16 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21819 @HyukjinKwon , can you take it forward? Appreciate your effort and thanks in advance. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22124 cc @gengliangwang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22117 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94851/ Test FAILed. ---

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22117 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22117: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22117 **[Test build #94851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94851/testReport)** for PR 22117 at commit

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22124 **[Test build #94861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94861/testReport)** for PR 22124 at commit

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22124: [SPARK-25135][SQL] Insert datasource table may all null ...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22124 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22124: [SPARK-25135][SQL] Insert datasource table may al...

2018-08-16 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22124 [SPARK-25135][SQL] Insert datasource table may all null when select from view ## What changes were proposed in this pull request? How to reproduce: ```scala val path =

[GitHub] spark issue #22081: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...

2018-08-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22081 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   >