[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136259136 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,65 @@ +/* + * Licen

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19092 **[Test build #81275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81275/testReport)** for PR 19092 at commit [`3dad127`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19092 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81275/ Test FAILed. ---

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19092 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136261767 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,68 @@ +/* + * Licen

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136261943 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,83 @@ class InsertIntoHiveTableSuite extends QueryTest w

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136262362 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,83 @@ class InsertIntoHiveTableSuite extends QueryTest w

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19092 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19092 **[Test build #81276 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81276/testReport)** for PR 19092 at commit [`3dad127`](https://github.com/apache/spark/commit/3d

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19086 **[Test build #81277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81277/testReport)** for PR 19086 at commit [`7e0954c`](https://github.com/apache/spark/commit/7e

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r136266187 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1001,6 +1001,16 @@ abstract class CodeGe

[GitHub] spark issue #18732: [SPARK-20396][SQL][PySpark] groupby().apply() with panda...

2017-08-31 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18732 there's actually a number of key people participating in the discussion in JIRA/SPIP, so I think we are good I think perhaps SPARK-20396 should be a subtask instead for tracking the ove

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-08-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r136274847 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tuning/CrossValidatorSuite.scala --- @@ -120,6 +120,33 @@ class CrossValidatorSuite }

[GitHub] spark pull request #19093: [SPARK-21880][web UI]In the SQL table page, modif...

2017-08-31 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19093#discussion_r136275068 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala --- @@ -177,7 +177,7 @@ private[ui] class RunningExecutionTable(

[GitHub] spark issue #19093: [SPARK-21880][web UI]In the SQL table page, modify jobs ...

2017-08-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19093 CC @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if th

[GitHub] spark pull request #19093: [SPARK-21880][web UI]In the SQL table page, modif...

2017-08-31 Thread Geek-He
Github user Geek-He commented on a diff in the pull request: https://github.com/apache/spark/pull/19093#discussion_r136279963 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala --- @@ -177,7 +177,7 @@ private[ui] class RunningExecutionTable

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18865 I think it makes sense to issue an error with good helpful message when users only select `_corrupt_record` without other columns. --- If your project is set up for it, you can reply to this email a

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19092 **[Test build #81276 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81276/testReport)** for PR 19092 at commit [`3dad127`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19092 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81276/ Test PASSed. ---

[GitHub] spark pull request #19094: [SPARK-21884][SQL] Fix StackOverflowError on Meta...

2017-08-31 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/19094 [SPARK-21884][SQL] Fix StackOverflowError on MetadataOnlyQuery ## What changes were proposed in this pull request? This PR aims to fix StackOverflowError in `branch-2.2`. This happens

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19086 **[Test build #81277 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81277/testReport)** for PR 19086 at commit [`7e0954c`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19086 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19094: [SPARK-21884][SQL] Fix StackOverflowError on MetadataOnl...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19094 **[Test build #81278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81278/testReport)** for PR 19094 at commit [`07126f7`](https://github.com/apache/spark/commit/07

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19086 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81277/ Test PASSed. ---

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19078 **[Test build #81279 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81279/testReport)** for PR 19078 at commit [`17d7aed`](https://github.com/apache/spark/commit/17

[GitHub] spark pull request #19095: [SPARK-21886][SQL] Use SparkSession.internalCreat...

2017-08-31 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/19095 [SPARK-21886][SQL] Use SparkSession.internalCreateDataFrame to create… … Dataset with LogicalRDD logical operator ## What changes were proposed in this pull request? Reu

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19095 **[Test build #81280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81280/testReport)** for PR 19095 at commit [`fff4b61`](https://github.com/apache/spark/commit/ff

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136305238 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136305819 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136306135 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136304803 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136311477 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19078 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19078 **[Test build #81279 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81279/testReport)** for PR 19078 at commit [`17d7aed`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19078 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81279/ Test FAILed. ---

[GitHub] spark pull request #19096: [SPARK-21869][SS] A cached Kafka producer should ...

2017-08-31 Thread ScrapCodes
GitHub user ScrapCodes opened a pull request: https://github.com/apache/spark/pull/19096 [SPARK-21869][SS] A cached Kafka producer should not be closed if any task is using it. ## What changes were proposed in this pull request? By updating the access time for the producer

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19096 **[Test build #81281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81281/testReport)** for PR 19096 at commit [`f21bf16`](https://github.com/apache/spark/commit/f2

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-31 Thread dm-tran
Github user dm-tran commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r136315990 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -114,7 +114,16 @@ class JsonFileFormat extend

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19096 **[Test build #81282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81282/testReport)** for PR 19096 at commit [`0afb48c`](https://github.com/apache/spark/commit/0a

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19096 **[Test build #81281 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81281/testReport)** for PR 19096 at commit [`f21bf16`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19096 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19096 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81281/ Test PASSed. ---

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19096 **[Test build #81282 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81282/testReport)** for PR 19096 at commit [`0afb48c`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19096 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81282/ Test PASSed. ---

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19096 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19094 **[Test build #81278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81278/testReport)** for PR 19094 at commit [`07126f7`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19094 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81278/ Test PASSed. ---

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19094 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19095 **[Test build #81280 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81280/testReport)** for PR 19095 at commit [`fff4b61`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19095 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81280/ Test PASSed. ---

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19095 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136332399 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2017-08-31 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r136332974 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/HeapMemoryAllocator.java --- @@ -47,23 +47,29 @@ private boolean shouldPool(long siz

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136333104 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-08-31 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18935 @squito can you please review again? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2017-08-31 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19077 This PR generally looks fine to me, my concern is that will this change bring in subtle impact on the code which leverage it. CC @JoshRosen to take a review. --- If your project is set u

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-08-31 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/17014 I think about this double-cache issue for a few days. One big problem is that, we are hard get precise storage level info. For example, we may add `map` transform on cached dataset and then pas

[GitHub] spark issue #18610: [SPARK-21386] ML LinearRegression supports warm start fr...

2017-08-31 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/18610 Thanks for the reply. Since there's already an agreement, I will hold my suggestion on initialModel data type. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #17461: [SPARK-20082][ml] LDA incremental model learning

2017-08-31 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17461#discussion_r135430463 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -32,10 +34,7 @@ import org.apache.spark.ml.param._ import org.apache.spark

[GitHub] spark pull request #17461: [SPARK-20082][ml] LDA incremental model learning

2017-08-31 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17461#discussion_r135430545 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -180,6 +179,29 @@ private[clustering] trait LDAParams extends Params with HasF

[GitHub] spark issue #19072: [SPARK-17139][ML][FOLLOW-UP] Add convenient method `asBi...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19072 **[Test build #81283 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81283/testReport)** for PR 19072 at commit [`e185d37`](https://github.com/apache/spark/commit/e1

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-31 Thread aray
Github user aray commented on the issue: https://github.com/apache/spark/pull/18818 ping @viirya @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #18306: [SPARK-21029][SS] All StreamingQuery should be stopped w...

2017-08-31 Thread aray
Github user aray commented on the issue: https://github.com/apache/spark/pull/18306 ping @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19094 Hi, @lianhuiwang and @hvanhovell . Could you review this PR? When this was introduced at 2.1.0, there was no problem. When the underlying classes of `fsRelation.location.listFiles` chan

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18697 **[Test build #81284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81284/testReport)** for PR 18697 at commit [`70a7268`](https://github.com/apache/spark/commit/70

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18697 **[Test build #81284 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81284/testReport)** for PR 18697 at commit [`70a7268`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18697 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81284/ Test FAILed. ---

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18697 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19078 **[Test build #81285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81285/testReport)** for PR 19078 at commit [`3304092`](https://github.com/apache/spark/commit/33

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18697 **[Test build #81286 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81286/testReport)** for PR 18697 at commit [`0f21237`](https://github.com/apache/spark/commit/0f

[GitHub] spark issue #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with the im...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18538 **[Test build #81287 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81287/testReport)** for PR 18538 at commit [`45d1380`](https://github.com/apache/spark/commit/45

[GitHub] spark issue #18731: [SPARK-20990][SQL] Read all JSON documents in files when...

2017-08-31 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/18731 AppVeyor build failed because of a timeout. I don't know how to retest it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136373078 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19092 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark pull request #18697: [SPARK-16683][SQL] Repeated joins to same table c...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18697#discussion_r136378843 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala --- @@ -171,6 +171,16 @@ sealed trait Partitioning

[GitHub] spark issue #18787: [SPARK-21583][SQL] Create a ColumnarBatch from ArrowColu...

2017-08-31 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/18787 Thanks @ueshin! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18270: [SPARK-21055][SQL] replace grouping__id with grouping_id...

2017-08-31 Thread cenyuhai
Github user cenyuhai commented on the issue: https://github.com/apache/spark/pull/18270 I can't see any comment at 77d4f7c? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #19065: [SPARK-21729][ML][TEST] Generic test for ProbabilisticCl...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19065 **[Test build #81288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81288/testReport)** for PR 19065 at commit [`f13cd73`](https://github.com/apache/spark/commit/f1

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-31 Thread rvesse
Github user rvesse commented on the issue: https://github.com/apache/spark/pull/18519 @ArtRand Any plans to add delegation token renewal under Mesos in the future? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19092 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark pull request #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestU...

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19092 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18270: [SPARK-21055][SQL] replace grouping__id with grouping_id...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18270 That commit is the code changes I suggested. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark issue #18270: [SPARK-21055][SQL] replace grouping__id with grouping_id...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18270 **[Test build #81289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81289/testReport)** for PR 18270 at commit [`2c6ed67`](https://github.com/apache/spark/commit/2c

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19078 **[Test build #81285 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81285/testReport)** for PR 19078 at commit [`3304092`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19078 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19078 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81285/ Test PASSed. ---

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19095 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19095 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark pull request #19095: [SPARK-21886][SQL] Use SparkSession.internalCreat...

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19095 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with the im...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18538 **[Test build #81287 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81287/testReport)** for PR 18538 at commit [`45d1380`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with the im...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18538 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81287/ Test PASSed. ---

[GitHub] spark issue #19089: [SPARK-21728][core] Follow up: fix user config, auth in ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19089 **[Test build #81290 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81290/testReport)** for PR 19089 at commit [`31d6c77`](https://github.com/apache/spark/commit/31

[GitHub] spark issue #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with the im...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18538 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81291 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81291/testReport)** for PR 18975 at commit [`b2068ce`](https://github.com/apache/spark/commit/b2

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread jaceklaskowski
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/19095 That was really quick! Thanks a lot @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #19065: [SPARK-21729][ML][TEST] Generic test for ProbabilisticCl...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19065 **[Test build #81288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81288/testReport)** for PR 19065 at commit [`f13cd73`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19065: [SPARK-21729][ML][TEST] Generic test for ProbabilisticCl...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19095 @jaceklaskowski Maybe you can fix the PR title next time. Thanks for your work! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #19065: [SPARK-21729][ML][TEST] Generic test for ProbabilisticCl...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81288/ Test PASSed. ---

  1   2   3   >