[GitHub] spark pull request #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestU...

2017-08-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19092 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19092 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-31 Thread rvesse
Github user rvesse commented on the issue: https://github.com/apache/spark/pull/18519 @ArtRand Any plans to add delegation token renewal under Mesos in the future? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #19065: [SPARK-21729][ML][TEST] Generic test for ProbabilisticCl...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19065 **[Test build #81288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81288/testReport)** for PR 19065 at commit

[GitHub] spark issue #18270: [SPARK-21055][SQL] replace grouping__id with grouping_id...

2017-08-31 Thread cenyuhai
Github user cenyuhai commented on the issue: https://github.com/apache/spark/pull/18270 I can't see any comment at 77d4f7c? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18787: [SPARK-21583][SQL] Create a ColumnarBatch from ArrowColu...

2017-08-31 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/18787 Thanks @ueshin! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #18697: [SPARK-16683][SQL] Repeated joins to same table c...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18697#discussion_r136378843 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala --- @@ -171,6 +171,16 @@ sealed trait

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19092 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136373078 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #18731: [SPARK-20990][SQL] Read all JSON documents in files when...

2017-08-31 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/18731 AppVeyor build failed because of a timeout. I don't know how to retest it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with the im...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18538 **[Test build #81287 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81287/testReport)** for PR 18538 at commit

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18697 **[Test build #81286 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81286/testReport)** for PR 18697 at commit

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19078 **[Test build #81285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81285/testReport)** for PR 19078 at commit

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18697 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18697 **[Test build #81284 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81284/testReport)** for PR 18697 at commit

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18697 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81284/ Test FAILed. ---

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18697 **[Test build #81284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81284/testReport)** for PR 18697 at commit

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19094 Hi, @lianhuiwang and @hvanhovell . Could you review this PR? When this was introduced at 2.1.0, there was no problem. When the underlying classes of `fsRelation.location.listFiles`

[GitHub] spark issue #18306: [SPARK-21029][SS] All StreamingQuery should be stopped w...

2017-08-31 Thread aray
Github user aray commented on the issue: https://github.com/apache/spark/pull/18306 ping @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-31 Thread aray
Github user aray commented on the issue: https://github.com/apache/spark/pull/18818 ping @viirya @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #19072: [SPARK-17139][ML][FOLLOW-UP] Add convenient method `asBi...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19072 **[Test build #81283 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81283/testReport)** for PR 19072 at commit

[GitHub] spark pull request #17461: [SPARK-20082][ml] LDA incremental model learning

2017-08-31 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17461#discussion_r135430545 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -180,6 +179,29 @@ private[clustering] trait LDAParams extends Params with

[GitHub] spark pull request #17461: [SPARK-20082][ml] LDA incremental model learning

2017-08-31 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17461#discussion_r135430463 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -32,10 +34,7 @@ import org.apache.spark.ml.param._ import

[GitHub] spark issue #18610: [SPARK-21386] ML LinearRegression supports warm start fr...

2017-08-31 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/18610 Thanks for the reply. Since there's already an agreement, I will hold my suggestion on initialModel data type. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-08-31 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/17014 I think about this double-cache issue for a few days. One big problem is that, we are hard get precise storage level info. For example, we may add `map` transform on cached dataset and then

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2017-08-31 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19077 This PR generally looks fine to me, my concern is that will this change bring in subtle impact on the code which leverage it. CC @JoshRosen to take a review. --- If your project is set

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-08-31 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18935 @squito can you please review again? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136333104 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2017-08-31 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r136332974 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/HeapMemoryAllocator.java --- @@ -47,23 +47,29 @@ private boolean shouldPool(long

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136332399 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19095 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19095 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81280/ Test PASSed. ---

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19095 **[Test build #81280 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81280/testReport)** for PR 19095 at commit

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19094 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19094 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81278/ Test PASSed. ---

[GitHub] spark issue #19094: [SPARK-21884][SQL][BRANCH-2.2] Fix StackOverflowError on...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19094 **[Test build #81278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81278/testReport)** for PR 19094 at commit

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19096 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19096 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81282/ Test PASSed. ---

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19096 **[Test build #81282 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81282/testReport)** for PR 19096 at commit

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19096 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81281/ Test PASSed. ---

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19096 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19096 **[Test build #81281 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81281/testReport)** for PR 19096 at commit

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19096 **[Test build #81282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81282/testReport)** for PR 19096 at commit

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-31 Thread dm-tran
Github user dm-tran commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r136315990 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -114,7 +114,16 @@ class JsonFileFormat

[GitHub] spark issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be ...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19096 **[Test build #81281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81281/testReport)** for PR 19096 at commit

[GitHub] spark pull request #19096: [SPARK-21869][SS] A cached Kafka producer should ...

2017-08-31 Thread ScrapCodes
GitHub user ScrapCodes opened a pull request: https://github.com/apache/spark/pull/19096 [SPARK-21869][SS] A cached Kafka producer should not be closed if any task is using it. ## What changes were proposed in this pull request? By updating the access time for the producer

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19078 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81279/ Test FAILed. ---

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19078 **[Test build #81279 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81279/testReport)** for PR 19078 at commit

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19078 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136311477 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136306135 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136304803 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136305238 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-31 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r136305819 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,379 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19095: [SPARK-21886][SQL] Use SparkSession.internalCreateDataFr...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19095 **[Test build #81280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81280/testReport)** for PR 19095 at commit

[GitHub] spark pull request #19095: [SPARK-21886][SQL] Use SparkSession.internalCreat...

2017-08-31 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/19095 [SPARK-21886][SQL] Use SparkSession.internalCreateDataFrame to create… … Dataset with LogicalRDD logical operator ## What changes were proposed in this pull request?

[GitHub] spark issue #19078: [SPARK-21862][ML] Add overflow check in PCA

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19078 **[Test build #81279 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81279/testReport)** for PR 19078 at commit

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19086 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81277/ Test PASSed. ---

[GitHub] spark issue #19094: [SPARK-21884][SQL] Fix StackOverflowError on MetadataOnl...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19094 **[Test build #81278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81278/testReport)** for PR 19094 at commit

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19086 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19086 **[Test build #81277 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81277/testReport)** for PR 19086 at commit

[GitHub] spark pull request #19094: [SPARK-21884][SQL] Fix StackOverflowError on Meta...

2017-08-31 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/19094 [SPARK-21884][SQL] Fix StackOverflowError on MetadataOnlyQuery ## What changes were proposed in this pull request? This PR aims to fix StackOverflowError in `branch-2.2`. This

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81276/ Test PASSed. ---

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19092 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19092 **[Test build #81276 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81276/testReport)** for PR 19092 at commit

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18865 I think it makes sense to issue an error with good helpful message when users only select `_corrupt_record` without other columns. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #19093: [SPARK-21880][web UI]In the SQL table page, modif...

2017-08-31 Thread Geek-He
Github user Geek-He commented on a diff in the pull request: https://github.com/apache/spark/pull/19093#discussion_r136279963 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala --- @@ -177,7 +177,7 @@ private[ui] class

[GitHub] spark issue #19093: [SPARK-21880][web UI]In the SQL table page, modify jobs ...

2017-08-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19093 CC @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #19093: [SPARK-21880][web UI]In the SQL table page, modif...

2017-08-31 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19093#discussion_r136275068 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala --- @@ -177,7 +177,7 @@ private[ui] class

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-08-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r136274847 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tuning/CrossValidatorSuite.scala --- @@ -120,6 +120,33 @@ class CrossValidatorSuite }

[GitHub] spark issue #18732: [SPARK-20396][SQL][PySpark] groupby().apply() with panda...

2017-08-31 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18732 there's actually a number of key people participating in the discussion in JIRA/SPIP, so I think we are good I think perhaps SPARK-20396 should be a subtask instead for tracking the

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r136266187 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1001,6 +1001,16 @@ abstract class

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19086 **[Test build #81277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81277/testReport)** for PR 19086 at commit

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19092 **[Test build #81276 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81276/testReport)** for PR 19092 at commit

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19092 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136262362 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,83 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136261943 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,83 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136261767 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,68 @@ +/* + *

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19092 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19092 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81275/ Test FAILed. ---

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19092 **[Test build #81275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81275/testReport)** for PR 19092 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136259136 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,65 @@ +/* + *

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136258558 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,118 @@ +/* + * Licensed to

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136258236 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -155,6 +156,9 @@ object HiveAnalysis extends Rule[LogicalPlan] {

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136258072 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -140,6 +141,9 @@ case class

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17980 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81274/ Test PASSed. ---

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17980 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17980 **[Test build #81274 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81274/testReport)** for PR 17980 at commit

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18975 Just left a comment : https://github.com/apache/spark/pull/18975#discussion_r136256422 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136256422 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1509,4 +1509,84 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #19093: [SPARK-21880][web UI]In the SQL table page, modify jobs ...

2017-08-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19093 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #19093: [SPARK-21880][web UI]In the SQL table page, modif...

2017-08-31 Thread Geek-He
GitHub user Geek-He opened a pull request: https://github.com/apache/spark/pull/19093 [SPARK-21880][web UI]In the SQL table page, modify jobs trace information ## What changes were proposed in this pull request? As shown below, for example, When the job 5 is running, It was a

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-08-31 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/19086 Thanks, I will refine soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19086 Test cases are missing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136252314 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClient.scala --- @@ -95,6 +95,9 @@ private[hive] trait HiveClient { /**

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136252223 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -576,25 +576,25 @@ class SessionCatalog(

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136252051 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala --- @@ -264,21 +264,22 @@ class InMemoryCatalog(

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-08-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136251887 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalog.scala --- @@ -131,13 +131,21 @@ abstract class

[GitHub] spark issue #19092: [SPARK-21878] [SQL] [TEST] Create SQLMetricsTestUtils

2017-08-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19092 **[Test build #81275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81275/testReport)** for PR 19092 at commit

<    1   2   3