[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18779 @maropu Because we already support it in Dataset API as the example https://github.com/apache/spark/pull/18779#issuecomment-319880753 shows, I'm afraid that there are users using this feature. If we

[GitHub] spark issue #18834: [SPARK-21618][DOCS] Note that http(s) not accepted in sp...

2017-08-03 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18834 With #18235 , I added Http(s) support for resources like files, jars. SparkSubmit will fetch them remotely and download to local tmp dir. --- If your project is set up for it, you can reply to

[GitHub] spark issue #18834: [SPARK-21618][DOCS] Note that http(s) not accepted in sp...

2017-08-03 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18834 I think https should be supported with this JIRA (https://github.com/apache/spark/pull/18235). --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #18834: [SPARK-21618][DOCS] Note that http(s) not accepted in sp...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18834 **[Test build #80209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80209/testReport)** for PR 18834 at commit

[GitHub] spark pull request #16158: [SPARK-18724][ML] Add TuningSummary for TrainVali...

2017-08-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/16158#discussion_r131133998 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tuning/CrossValidatorSuite.scala --- @@ -66,6 +66,29 @@ class CrossValidatorSuite

[GitHub] spark issue #16158: [SPARK-18724][ML] Add TuningSummary for TrainValidationS...

2017-08-03 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/16158 @hhbyyh sorry for the delay. Left a few review comments. Tested the examples and it looks cool! Very useful --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #16158: [SPARK-18724][ML] Add TuningSummary for TrainVali...

2017-08-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/16158#discussion_r131133851 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/TuningSummary.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16158: [SPARK-18724][ML] Add TuningSummary for TrainVali...

2017-08-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/16158#discussion_r131133018 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -133,7 +134,10 @@ class CrossValidator @Since("1.2.0")

[GitHub] spark pull request #16158: [SPARK-18724][ML] Add TuningSummary for TrainVali...

2017-08-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/16158#discussion_r131132815 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -229,6 +233,29 @@ class CrossValidatorModel private[ml] (

[GitHub] spark pull request #16158: [SPARK-18724][ML] Add TuningSummary for TrainVali...

2017-08-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/16158#discussion_r131132649 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/TrainValidationSplit.scala --- @@ -231,6 +235,29 @@ class TrainValidationSplitModel private[ml] (

[GitHub] spark pull request #16158: [SPARK-18724][ML] Add TuningSummary for TrainVali...

2017-08-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/16158#discussion_r131133463 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/TuningSummary.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #18834: [SPARK-21618][DOCS] Note that http(s) not accepte...

2017-08-03 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/18834 [SPARK-21618][DOCS] Note that http(s) not accepted in spark-submit jar uri ## What changes were proposed in this pull request? Remove https from list of supported URIs, until it's

[GitHub] spark issue #18630: [SPARK-12559][SPARK SUBMIT] fix --packages for stand-alo...

2017-08-03 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/18630 @vanzin fixed the issues. Please give it another try or merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #18630: [SPARK-12559][SPARK SUBMIT] fix --packages for stand-alo...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18630 **[Test build #80208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80208/testReport)** for PR 18630 at commit

[GitHub] spark issue #18807: [SPARK-21601][BUILD] Modify the pom.xml file, increase t...

2017-08-03 Thread highfei2011
Github user highfei2011 commented on the issue: https://github.com/apache/spark/pull/18807 Hi, @srowen @markhamstra ,This problem has been solved,[https://github.com/apache/spark/pull/18808](url) --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #18804: [SPARK-21599][SQL] Collecting column statistics for data...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18804 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18804: [SPARK-21599][SQL] Collecting column statistics for data...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18804 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80204/ Test PASSed. ---

[GitHub] spark issue #18804: [SPARK-21599][SQL] Collecting column statistics for data...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18804 **[Test build #80204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80204/testReport)** for PR 18804 at commit

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17357 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17357 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80202/ Test PASSed. ---

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17357 **[Test build #80202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80202/testReport)** for PR 17357 at commit

[GitHub] spark issue #18783: [SPARK-21254] [WebUI] History UI performance fixes

2017-08-03 Thread 2ooom
Github user 2ooom commented on the issue: https://github.com/apache/spark/pull/18783 Thank you @srowen. Seems like jenkins is happy. Should we merge in this case? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18313: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-08-03 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18313 I commented on the [JIRA](https://issues.apache.org/jira/browse/SPARK-21086?focusedCommentId=16112623=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16112623). I

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-03 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r131121892 --- Diff: mllib/src/test/scala/org/apache/spark/ml/evaluation/ClusteringEvaluatorSuite.scala --- @@ -0,0 +1,235 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18733: [SPARK-21535][ML]Reduce memory requirement for Cr...

2017-08-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/18733#discussion_r131121125 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -112,16 +112,16 @@ class CrossValidator @Since("1.2.0")

[GitHub] spark issue #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an emergen...

2017-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18797 I've figured out the problem, and pretty sure it's a problem in the AFT test that was hidden until now. It runs AFTSurvivlaRegression on this input: ``` ++-+--+--+

[GitHub] spark issue #18802: [SPARK-18535][SPARK-19720][CORE][BACKPORT-2.1] Redact se...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18802 **[Test build #80207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80207/testReport)** for PR 18802 at commit

[GitHub] spark issue #18833: [SPARK-21625][SQL] sqrt(negative number) should be null.

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18833 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80200/ Test PASSed. ---

[GitHub] spark issue #18833: [SPARK-21625][SQL] sqrt(negative number) should be null.

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18833 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18779 @viirya btw, we still need to support group/sort-by-ordinal in Dataset? If this pr merged, the behaviour seems to change. I feel the syntax in Dataset is a little confusing.. --- If your project

[GitHub] spark issue #18833: [SPARK-21625][SQL] sqrt(negative number) should be null.

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18833 **[Test build #80200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80200/testReport)** for PR 18833 at commit

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18779 @viirya great! thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18779 **[Test build #80206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80206/testReport)** for PR 18779 at commit

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/18779 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #18808: [SPARK-21605][BUILD] Let IntelliJ IDEA correctly ...

2017-08-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18808 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18808: [SPARK-21605][BUILD] Let IntelliJ IDEA correctly detect ...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18808 **[Test build #3874 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3874/testReport)** for PR 18808 at commit

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14151 **[Test build #80205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80205/testReport)** for PR 14151 at commit

[GitHub] spark issue #18815: [SPARK-21609][WEB-UI]In the Master ui add "log directory...

2017-08-03 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/18815 Yes, I am also considering this security issue. If this only shows a path, only authorized people can log on to the spark cluster lunix server for log view, which is relatively safe. ---

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14151 **[Test build #80203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80203/testReport)** for PR 14151 at commit

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14151 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80203/ Test FAILed. ---

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14151 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18815: [SPARK-21609][WEB-UI]In the Master ui add "log directory...

2017-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18815 I don't know if this is that important, so not sure if it's worth much code. Would people generally be able to access the logs anyway? is there a security issue if I can see the masters logs and

[GitHub] spark issue #18815: [SPARK-21609][WEB-UI]In the Master ui add "log directory...

2017-08-03 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/18815 Ok, i will work it, just like hbase. Show master and worker logs, and provide online viewing. Is this okay? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an emergen...

2017-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18797 The only number that is <= Double.PositiveInfinity is Double.NaN, because it has no ordering at all with respect to anything. So init must be NaN somehow. It's called from LBFGS in Breeze,

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18779 @maropu The PR is at #17770. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18815: [SPARK-21609][WEB-UI]In the Master ui add "log directory...

2017-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18815 OK, then how can this work? this won't render as a usable link. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #18815: [SPARK-21609][WEB-UI]In the Master ui add "log directory...

2017-08-03 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/18815 Yes, it is not an http URI ,at present only a path to tell the user log path where to facilitate the lunix server query related log information. --- If your project is set up for it, you

[GitHub] spark issue #18804: [SPARK-21599][SQL] Collecting column statistics for data...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18804 **[Test build #80204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80204/testReport)** for PR 18804 at commit

[GitHub] spark issue #18281: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-08-03 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18281 I also think we can leave any potential improvements for parallelism on the Python side (as well as the test side if we come up with a good way of testing that fitting is actually being done in

[GitHub] spark issue #18281: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-08-03 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18281 @ajaysaini725 Could you resolve merge conflicts and address the remaining outstanding review comments? I left a few minor additional comments. Overall I think this is just about ready.

[GitHub] spark pull request #18281: [SPARK-21027][ML][PYTHON] Added tunable paralleli...

2017-08-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/18281#discussion_r131104871 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/OneVsRestSuite.scala --- @@ -101,6 +101,44 @@ class OneVsRestSuite extends SparkFunSuite

[GitHub] spark pull request #18281: [SPARK-21027][ML][PYTHON] Added tunable paralleli...

2017-08-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/18281#discussion_r131104233 --- Diff: python/pyspark/ml/classification.py --- @@ -1517,20 +1518,23 @@ class OneVsRest(Estimator, OneVsRestParams, JavaMLReadable, JavaMLWritable):

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14151 **[Test build #80203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80203/testReport)** for PR 14151 at commit

[GitHub] spark pull request #18816: [SPARK-21611][SQL]Error class name for log in sev...

2017-08-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18816 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18815: [SPARK-21609][WEB-UI]In the Master ui add "log directory...

2017-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18815 It's not an http URI though right? it's a path. I'm missing why this is browseable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #18816: [SPARK-21611][SQL]Error class name for log in several cl...

2017-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18816 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #18281: [SPARK-21027][ML][PYTHON] Added tunable paralleli...

2017-08-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/18281#discussion_r131102727 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala --- @@ -294,6 +296,18 @@ final class OneVsRest @Since("1.4.0") (

[GitHub] spark issue #18829: [SPARK-21620][WEB-UI][CORE]Add metrics url in spark web ...

2017-08-03 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/18829 @srowen @ajbozarth Help review the code,Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #18815: [SPARK-21609][WEB-UI]In the Master ui add "log directory...

2017-08-03 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/18815 1.The master process information is as follows: /usr/java8/jdk/bin/java -cp

[GitHub] spark issue #18668: [SPARK-21451][SQL]get `spark.hadoop.*` properties from s...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18668 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80198/ Test PASSed. ---

[GitHub] spark issue #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with the im...

2017-08-03 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18538 @gatorsmile Could you help to trigger the test job? It seems I can't do it now. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18668: [SPARK-21451][SQL]get `spark.hadoop.*` properties from s...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18668 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18668: [SPARK-21451][SQL]get `spark.hadoop.*` properties from s...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18668 **[Test build #80198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80198/testReport)** for PR 18668 at commit

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-03 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r131100309 --- Diff: mllib/src/test/scala/org/apache/spark/ml/evaluation/ClusteringEvaluatorSuite.scala --- @@ -0,0 +1,235 @@ +/* + * Licensed to the

[GitHub] spark pull request #14151: [SPARK-16496][SQL] Add wholetext as option for re...

2017-08-03 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/14151#discussion_r131100257 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileWholeTextReader.scala --- @@ -0,0 +1,57 @@ +/* + *

[GitHub] spark issue #18816: [SPARK-21611][SQL]Error class name for log in several cl...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18816 **[Test build #3876 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3876/testReport)** for PR 18816 at commit

[GitHub] spark issue #18832: [SPARK-21623][ML]fix RF doc

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18832 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80199/ Test PASSed. ---

[GitHub] spark issue #18832: [SPARK-21623][ML]fix RF doc

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18832 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18628: [SPARK-18061][ThriftServer] Add spnego auth support for ...

2017-08-03 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/18628 Thanks for making sure this is consistent with other uses of Configuration.get(); consistency is critical here --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #18832: [SPARK-21623][ML]fix RF doc

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18832 **[Test build #80199 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80199/testReport)** for PR 18832 at commit

[GitHub] spark issue #18815: [SPARK-21609][WEB-UI]In the Master ui add "log directory...

2017-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18815 Hm, I thought so, even in standalone mode, but I don't know. OK, if it's a file URI, is that going to be browseable? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #18668: [SPARK-21451][SQL]get `spark.hadoop.*` properties...

2017-08-03 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/18668#discussion_r131095350 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala --- @@ -50,6 +50,7 @@ private[hive]

[GitHub] spark issue #18528: [SPARK-13041][Mesos] Adds sandbox uri to spark dispatche...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18528 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18528: [SPARK-13041][Mesos] Adds sandbox uri to spark dispatche...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18528 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80201/ Test PASSed. ---

[GitHub] spark issue #18528: [SPARK-13041][Mesos] Adds sandbox uri to spark dispatche...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18528 **[Test build #80201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80201/testReport)** for PR 18528 at commit

[GitHub] spark pull request #18668: [SPARK-21451][SQL]get `spark.hadoop.*` properties...

2017-08-03 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/18668#discussion_r131094720 --- Diff: docs/configuration.md --- @@ -2335,5 +2335,61 @@ The location of these configuration files varies across Hadoop versions, but a common

[GitHub] spark issue #18815: [SPARK-21609][WEB-UI]In the Master ui add "log directory...

2017-08-03 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/18815 @srowen Are you sure that the master and worker logs can be stored in hdfs? Spark's master and worker logs are generated by log4j. --- If your project is set up for it, you can

[GitHub] spark pull request #18668: [SPARK-21451][SQL]get `spark.hadoop.*` properties...

2017-08-03 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/18668#discussion_r131093892 --- Diff: docs/configuration.md --- @@ -2335,5 +2335,61 @@ The location of these configuration files varies across Hadoop versions, but a common

[GitHub] spark pull request #18668: [SPARK-21451][SQL]get `spark.hadoop.*` properties...

2017-08-03 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/18668#discussion_r131093320 --- Diff: docs/configuration.md --- @@ -2335,5 +2335,61 @@ The location of these configuration files varies across Hadoop versions, but a common

[GitHub] spark issue #18804: [SPARK-21599][SQL] Collecting column statistics for data...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18804 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17357 **[Test build #80202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80202/testReport)** for PR 17357 at commit

[GitHub] spark issue #18804: [SPARK-21599][SQL] Collecting column statistics for data...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18804 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80196/ Test PASSed. ---

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18779 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80197/ Test FAILed. ---

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18779 **[Test build #80197 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80197/testReport)** for PR 18779 at commit

[GitHub] spark issue #18804: [SPARK-21599][SQL] Collecting column statistics for data...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18804 **[Test build #80196 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80196/testReport)** for PR 18804 at commit

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18779 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18779 ok, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18528: [SPARK-13041][Mesos] Adds sandbox uri to spark dispatche...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18528 **[Test build #80201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80201/testReport)** for PR 18528 at commit

[GitHub] spark issue #18528: [SPARK-13041][Mesos] Adds sandbox uri to spark dispatche...

2017-08-03 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/18528 @srowen fixed, ready for merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18811: [SPARK-21604][SQL] if the object extends Logging,...

2017-08-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18811 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18816: [SPARK-21611][SQL]Error class name for log in several cl...

2017-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18816 **[Test build #3876 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3876/testReport)** for PR 18816 at commit

[GitHub] spark issue #18745: [SPARK-21544][DEPLOY] Tests jar of some module should no...

2017-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18745 Ping @caneGuy -- adding `[test-maven]` will let us also verify this passes the Maven build --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18833: [SPARK-21625][SQL] sqrt(negative number) should be null.

2017-08-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18833 Pg does not accept this query; ``` postgres=# select sqrt(3); sqrt -- 1.73205080756888 (1 row) postgres=# select sqrt(-1); ERROR:

[GitHub] spark issue #18811: [SPARK-21604][SQL] if the object extends Logging, i sugg...

2017-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18811 merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18779 @maropu I have a PR before to solve that. Due to some reasons it will be merged on 2.3. I am out of laptop, will refer it once I can access laptop. On Aug 3, 2017 5:07 PM,

[GitHub] spark issue #18811: [SPARK-21604][SQL] if the object extends Logging, i sugg...

2017-08-03 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/18811 ok, have done. Thanks @srowen . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18779: [SPARK-21580][SQL]Integers in aggregation expressions ar...

2017-08-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18779 @gatorsmile @viirya I looked into why it applied some analyzer rules into already-analyzed plans and I noticed that some rules used `transform/transformUp` instead of `resolveOperators` in `apply`.

[GitHub] spark issue #18607: [SPARK-21362][SQL][Adding Apache Drill JDBC Dialect]

2017-08-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18607 @radford1 Have you added the test as suggested by @gatorsmile? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #18832: [SPARK-21623][ML]fix RF doc

2017-08-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18832 @sethah --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #18821: [SPARK-21615][ML][MLlib][DOCS] Fix broken redirec...

2017-08-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18821 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

<    1   2   3   4   5   >