[GitHub] spark issue #15292: [SPARK-17719][SPARK-17776][SQL] Unify and tie up options...

2016-10-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15292 BTW, I finished the review. LGTM except the above comments. Let @cloud-fan double check it. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark issue #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelationProvid...

2016-10-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15263 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82337939 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,339 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark issue #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelationProvid...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15263 **[Test build #66489 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66489/consoleFull)** for PR 15263 at commit [`b339492`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14538: [SPARK-16960][SQL] Deprecate approxCountDistinct, toDegr...

2016-10-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14538 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark pull request #15292: [SPARK-17719][SPARK-17776][SQL] Unify and tie up ...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15292#discussion_r82339188 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -17,47 +17,130 @@ package org.apac

[GitHub] spark pull request #15381: [SPARK-17707] [WEBUI] Web UI prevents spark-submi...

2016-10-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15381#discussion_r82339491 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java --- @@ -90,8 +95,21 @@ public void run() {

[GitHub] spark issue #14538: [SPARK-16960][SQL] Deprecate approxCountDistinct, toDegr...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14538 **[Test build #66490 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66490/consoleFull)** for PR 14538 at commit [`bd5b379`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #15382: [SPARK-17810] [SQL] Default spark.sql.warehouse.dir is r...

2016-10-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15382 Good point. The docs all say it should be working dir, so I was following the apparent intent along the way here. I don't know if one is more surprising than the other. --- If your project is set

[GitHub] spark issue #15379: [SPARK-17805][PYSPARK] Fix in sqlContext.read.text when ...

2016-10-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15379 Thanks - the argument renaming issue is largely orthogonal and I don't think we can break it now. I'm going to merge this in master/2.0. --- If your project is set up for it, you can reply to this em

[GitHub] spark issue #15292: [SPARK-17719][SPARK-17776][SQL] Unify and tie up options...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15292 **[Test build #66491 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66491/consoleFull)** for PR 15292 at commit [`a749409`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #15379: [SPARK-17805][PYSPARK] Fix in sqlContext.read.tex...

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15379 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82340840 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,339 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark issue #15343: [SPARK-8480] [CORE] [PYSPARK] [SPARKR] Add setName for D...

2016-10-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15343 Hm, I think I'm going to close this for now because it doesn't actually fix the issue I thought it would, and I honestly could not trace how the cached Dataset logical plan becomes RDD info in the UI

[GitHub] spark pull request #15343: [SPARK-8480] [CORE] [PYSPARK] [SPARKR] Add setNam...

2016-10-07 Thread srowen
Github user srowen closed the pull request at: https://github.com/apache/spark/pull/15343 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14957: [SPARK-4502][SQL]Support parquet nested struct pruning a...

2016-10-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14957 This would be good to include in Spark 2.1... cc @davies and @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15370 Yeah, the name should be zero-padded to 10 digits to make the lexical ordering match the intended ordering. It may not matter, but seems a bit safer. OK right the code path doesn't just list

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82342444 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,339 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82342839 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,339 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82343881 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-07 Thread seyfe
Github user seyfe commented on the issue: https://github.com/apache/spark/pull/15371 Hi @srowen, This PR doesn't introduce any extra data copy operations. It moves the data copy code from `JsonProtocol:accumValueToJson` method to `BlockStatusesAccumulator:value` method so it

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82348471 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryStatus.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66492 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66492/consoleFull)** for PR 15307 at commit [`1655c96`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15307 @marmbrus - Refactored APIs (see description) - Added source-specific tests - Fixed LocalTableScanExec.executeCollect/executeTake not providing SQLMetrics - Reverted *Status classes to

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14638 **[Test build #66493 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66493/consoleFull)** for PR 14638 at commit [`ec85638`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #15355: [SPARK-17782][STREAMING][BUILD] Add Kafka 0.10 project t...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15355 **[Test build #66487 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66487/consoleFull)** for PR 15355 at commit [`a2ba414`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15355: [SPARK-17782][STREAMING][BUILD] Add Kafka 0.10 project t...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66487/ Test PASSed. ---

[GitHub] spark issue #15355: [SPARK-17782][STREAMING][BUILD] Add Kafka 0.10 project t...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15355 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15388: [SPARK-17821][SQL] Support And and Or in Expression Cano...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15388 **[Test build #66488 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66488/consoleFull)** for PR 15388 at commit [`0a48d86`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15388: [SPARK-17821][SQL] Support And and Or in Expression Cano...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15388 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15388: [SPARK-17821][SQL] Support And and Or in Expression Cano...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15388 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66488/ Test PASSed. ---

[GitHub] spark pull request #15239: [SPARK-17665][SPARKR] Support options/mode all fo...

2016-10-07 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15239#discussion_r82352445 --- Diff: R/pkg/R/SQLContext.R --- @@ -341,11 +342,13 @@ setMethod("toDF", signature(x = "RDD"), #' @name read.json #' @method read.json default

[GitHub] spark issue #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelationProvid...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15263 **[Test build #66489 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66489/consoleFull)** for PR 15263 at commit [`b339492`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelationProvid...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15263 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66489/ Test PASSed. ---

[GitHub] spark issue #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelationProvid...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15263 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15239: [SPARK-17665][SPARKR] Support options/mode all fo...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15239#discussion_r82355751 --- Diff: R/pkg/R/SQLContext.R --- @@ -341,11 +342,13 @@ setMethod("toDF", signature(x = "RDD"), #' @name read.json #' @method read.json default

[GitHub] spark issue #14538: [SPARK-16960][SQL] Deprecate approxCountDistinct, toDegr...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14538 **[Test build #66490 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66490/consoleFull)** for PR 14538 at commit [`bd5b379`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14538: [SPARK-16960][SQL] Deprecate approxCountDistinct, toDegr...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14538 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14538: [SPARK-16960][SQL] Deprecate approxCountDistinct, toDegr...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14538 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66490/ Test PASSed. ---

[GitHub] spark issue #15292: [SPARK-17719][SPARK-17776][SQL] Unify and tie up options...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15292 **[Test build #66491 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66491/consoleFull)** for PR 15292 at commit [`a749409`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15292: [SPARK-17719][SPARK-17776][SQL] Unify and tie up options...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15292 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15292: [SPARK-17719][SPARK-17776][SQL] Unify and tie up options...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15292 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66491/ Test PASSed. ---

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15371 I see the new copy (of course or else this wouldn't help) but where is a copy removed? I'm probably overlooking it. A `toList` call was moved, which we discussed above briefly. I still think it's all

[GitHub] spark pull request #15239: [SPARK-17665][SPARKR] Support options/mode all fo...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15239#discussion_r82362700 --- Diff: R/pkg/R/SQLContext.R --- @@ -341,11 +342,13 @@ setMethod("toDF", signature(x = "RDD"), #' @name read.json #' @method read.json default

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14638 **[Test build #66493 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66493/consoleFull)** for PR 14638 at commit [`ec85638`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14638 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14638 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66493/ Test PASSed. ---

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66492/consoleFull)** for PR 15307 at commit [`1655c96`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15307 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15307 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66492/ Test FAILed. ---

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66494 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66494/consoleFull)** for PR 15307 at commit [`3984d54`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66494 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66494/consoleFull)** for PR 15307 at commit [`3984d54`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15307 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66494/ Test FAILed. ---

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15307 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15355: [SPARK-17782][STREAMING][BUILD] Add Kafka 0.10 project t...

2016-10-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15355 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66495 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66495/consoleFull)** for PR 15307 at commit [`7efea10`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #15369: [SPARK-17795] [Web UI] Sorting on stage or job tables do...

2016-10-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15369 merged to master. Seems like a clean improvement. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request #15239: [SPARK-17665][SPARKR] Support options/mode all fo...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15239#discussion_r82367394 --- Diff: R/pkg/R/SQLContext.R --- @@ -341,11 +342,13 @@ setMethod("toDF", signature(x = "RDD"), #' @name read.json #' @method read.json default

[GitHub] spark pull request #15355: [SPARK-17782][STREAMING][BUILD] Add Kafka 0.10 pr...

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15355 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15369: [SPARK-17795] [Web UI] Sorting on stage or job ta...

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15369 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66495 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66495/consoleFull)** for PR 15307 at commit [`7efea10`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15307 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15307 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66495/ Test FAILed. ---

[GitHub] spark issue #14538: [SPARK-16960][SQL] Deprecate approxCountDistinct, toDegr...

2016-10-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14538 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark pull request #15366: [SPARK-17793] [Web UI] Sorting on the description...

2016-10-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15366#discussion_r82367835 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -462,7 +462,10 @@ private[ui] class JobDataSource( } case

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66496 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66496/consoleFull)** for PR 15307 at commit [`5e0aa57`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #14538: [SPARK-16960][SQL] Deprecate approxCountDistinct, toDegr...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14538 Thank you @srowen. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark issue #15361: [SPARK-17765][SQL] Support for writing out user-defined ...

2016-10-07 Thread kxepal
Github user kxepal commented on the issue: https://github.com/apache/spark/pull/15361 @HyukjinKwon It works great! Thank you! My mistake was by applying changes for the same `wrapperFor` method, while for 2.0.0 sources state it have to be placed in `wrap` method instead with a b

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-07 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/15346 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark pull request #15387: [SPARK-17782][STREAMING][KAFKA] eliminate race co...

2016-10-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15387#discussion_r82370539 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala --- @@ -223,7 +223,7 @@ private[spark] class Dir

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15346 **[Test build #66497 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66497/consoleFull)** for PR 15346 at commit [`605aeb6`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Str...

2016-10-07 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/14087#discussion_r82371339 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala --- @@ -378,6 +378,24 @@ class FileStreamSourceSuite extends

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14087 **[Test build #66498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66498/consoleFull)** for PR 14087 at commit [`867394d`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #15387: [SPARK-17782][STREAMING][KAFKA] eliminate race condition...

2016-10-07 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15387 I'm not going to say anything is impossible, which is the point of the assert. If it does somehow happen, it will be at start, so should be obvious. The whole poll 0 / pause thing i

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r82372368 --- Diff: core/src/main/scala/org/apache/spark/util/logging/RollingFileAppender.scala --- @@ -76,15 +79,40 @@ private[spark] class RollingFileAppender(

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r82373810 --- Diff: core/src/test/scala/org/apache/spark/util/FileAppenderSuite.scala --- @@ -292,8 +332,21 @@ class FileAppenderSuite extends SparkFunSuite with BeforeA

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r82373245 --- Diff: core/src/main/scala/org/apache/spark/util/logging/RollingFileAppender.scala --- @@ -142,6 +170,9 @@ private[spark] object RollingFileAppender {

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r82373348 --- Diff: core/src/test/scala/org/apache/spark/util/FileAppenderSuite.scala --- @@ -72,6 +74,24 @@ class FileAppenderSuite extends SparkFunSuite with BeforeAnd

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r82378093 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1455,7 +1456,11 @@ private[spark] object Utils extends Logging { val effectiveEn

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r82373610 --- Diff: core/src/test/scala/org/apache/spark/util/FileAppenderSuite.scala --- @@ -89,6 +109,25 @@ class FileAppenderSuite extends SparkFunSuite with BeforeAn

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r82373640 --- Diff: core/src/test/scala/org/apache/spark/util/FileAppenderSuite.scala --- @@ -89,6 +109,25 @@ class FileAppenderSuite extends SparkFunSuite with BeforeAn

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r82372648 --- Diff: core/src/main/scala/org/apache/spark/util/logging/RollingFileAppender.scala --- @@ -76,15 +79,40 @@ private[spark] class RollingFileAppender(

[GitHub] spark issue #14263: [SPARK-16626][PYTHON][MLLIB] Code duplication after SPAR...

2016-10-07 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/14263 Closing following [exchange on JIRA](https://issues.apache.org/jira/browse/SPARK-16626?focusedCommentId=15543929&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15543929

[GitHub] spark pull request #14263: [SPARK-16626][PYTHON][MLLIB] Code duplication aft...

2016-10-07 Thread zero323
Github user zero323 closed the pull request at: https://github.com/apache/spark/pull/14263 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark pull request #13324: [SPARK-15559][PYTHON][STREAMING] Add hash method ...

2016-10-07 Thread zero323
Github user zero323 closed the pull request at: https://github.com/apache/spark/pull/13324 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark issue #12578: [SPARK-10496][SQL] Add DataFrame cumulative sum

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12578 **[Test build #66499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66499/consoleFull)** for PR 12578 at commit [`97af903`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #14248: [SPARK-16589][PYTHON] Chained cartesian produces ...

2016-10-07 Thread zero323
Github user zero323 closed the pull request at: https://github.com/apache/spark/pull/14248 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66496 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66496/consoleFull)** for PR 15307 at commit [`5e0aa57`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15307 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66496/ Test FAILed. ---

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15307 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15370 I don't consider this complex, its simply one number formatter and you say get. Padding to 10 digits isn't necessary in 99% of the cases, so you are just using longer file strings and more digit

[GitHub] spark pull request #15370: [SPARK-17417][Core] Fix # of partitions for Relia...

2016-10-07 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15370#discussion_r82386685 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -71,6 +72,11 @@ private[spark] object CallSite { private[spark] object Utils ext

[GitHub] spark pull request #15370: [SPARK-17417][Core] Fix # of partitions for Relia...

2016-10-07 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15370#discussion_r82387014 --- Diff: core/src/main/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala --- @@ -96,16 +96,15 @@ private[spark] class ReliableCheckpointRDD[T: ClassT

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14087 **[Test build #66498 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66498/consoleFull)** for PR 14087 at commit [`867394d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14087 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66498/ Test PASSed. ---

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14087 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15370 If you still believe this to complex can you please clarify why so we can discuss. This is basically 5 lines of code (yes lines of code aren't good complexity indicator but..), 3 of them to create

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15346 **[Test build #66497 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66497/consoleFull)** for PR 15346 at commit [`605aeb6`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15346 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15346 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66497/ Test PASSed. ---

  1   2   3   4   5   6   7   >