[GitHub] spark pull request: [SPARK-10966] Codegen framework cleanup

2015-10-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9006#discussion_r41351532 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -384,11 +387,23 @@ abstract class

[GitHub] spark pull request: [SPARK-10917] [SQL] improve performance of com...

2015-10-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/8971#discussion_r41352348 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnStats.scala --- @@ -254,7 +255,7 @@ private[sql] class

[GitHub] spark pull request: [SPARK-10966] Codegen framework cleanup

2015-10-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9006#discussion_r41351350 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -112,21 +115,21 @@ class CodeGenContext

[GitHub] spark pull request: [SPARK-10412][SQL] report memory usage for tun...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8931#issuecomment-146080586 [Test build #43314 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43314/console) for PR 8931 at commit

[GitHub] spark pull request: [SPARK-10412][SQL] report memory usage for tun...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8931#issuecomment-146080646 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10412][SQL] report memory usage for tun...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8931#issuecomment-146080647 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-06 Thread Lewuathe
Github user Lewuathe commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-146080738 @dbtsai Thank you so much for caching up. Currently this patch fixed the latest issues based on your pointing out. Test failures does not seem to be related to this

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146078080 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146078070 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10966] Codegen framework cleanup

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9006#issuecomment-146077642 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8654][SQL] Fix Analysis exception when ...

2015-10-06 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/8983#issuecomment-146077599 if `null in (true, array(2,3))` returns null in hive, then we probably should first cast `key` and all values in `list` to a common type. If there is no such a common

[GitHub] spark pull request: [SPARK-10966] Codegen framework cleanup

2015-10-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9006#issuecomment-146077592 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-10966] Codegen framework cleanup

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9006#issuecomment-146077653 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10917] [SQL] improve performance of com...

2015-10-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/8971#discussion_r41352948 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnType.scala --- @@ -399,24 +409,158 @@ private[sql] object BINARY extends

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146080129 [Test build #43320 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43320/console) for PR 8869 at commit

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146080183 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146080181 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10917] [SQL] improve performance of com...

2015-10-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/8971#discussion_r41352081 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnType.scala --- @@ -399,24 +409,158 @@ private[sql] object BINARY extends

[GitHub] spark pull request: [SPARK-10917] [SQL] improve performance of com...

2015-10-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/8971#discussion_r41353513 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnType.scala --- @@ -365,32 +374,33 @@ private[sql] case class FIXED_DECIMAL(precision:

[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8967#issuecomment-146077350 [Test build #1849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1849/console) for PR 8967 at commit

[GitHub] spark pull request: [SPARK-10966] Codegen framework cleanup

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9006#issuecomment-146078196 [Test build #43319 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43319/consoleFull) for PR 9006 at commit

[GitHub] spark pull request: [SPARK-10966] Codegen framework cleanup

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9006#issuecomment-146079626 [Test build #43319 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43319/console) for PR 9006 at commit

[GitHub] spark pull request: [SPARK-10966] Codegen framework cleanup

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9006#issuecomment-146079646 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10966] Codegen framework cleanup

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9006#issuecomment-146079647 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

2015-10-06 Thread koertkuipers
Github user koertkuipers commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-145902167 goals (copied over from SPARK-5741 comments by @marmbrus ): It was originally just parquet that would support more than one file, but now all HadoopFSRelations

[GitHub] spark pull request: [SPARK-10938] [SQL] remove typeId in columnar ...

2015-10-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8989 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10879] spark on yarn support priority o...

2015-10-06 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/8943#issuecomment-145873474 Container priority is different. container priority is honored right now and its for the application to say one container must be allocated before another. An

[GitHub] spark pull request: [SPARK-10326] [yarn] Fix app submission on win...

2015-10-06 Thread jamartinh
Github user jamartinh commented on the pull request: https://github.com/apache/spark/pull/8493#issuecomment-145908478 I have tested it on windows with spark 1.5.0 and 1.5.1 and the problem still remains. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146078664 [Test build #43320 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43320/consoleFull) for PR 8869 at commit

[GitHub] spark pull request: [SPARK-10917] [SQL] improve performance of com...

2015-10-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/8971#discussion_r41352041 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnType.scala --- @@ -399,24 +409,158 @@ private[sql] object BINARY extends

[GitHub] spark pull request: [SPARK-10964] [yarn] Correctly register the AM...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9005#issuecomment-146066362 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10964] [yarn] Correctly register the AM...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9005#issuecomment-146066217 [Test build #43307 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43307/console) for PR 9005 at commit

[GitHub] spark pull request: [SPARK-10964] [yarn] Correctly register the AM...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9005#issuecomment-146066357 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10966] Codegen framework cleanup

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9006#issuecomment-146067798 [Test build #1848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1848/console) for PR 9006 at commit

[GitHub] spark pull request: [SPARK-10337][SQL] fix hive views on non-hive-...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8990#issuecomment-146067832 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10941] [SQL] Refactor AggregateFunction...

2015-10-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/8973#discussion_r41348104 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala --- @@ -95,98 +92,192 @@ private[sql] case

[GitHub] spark pull request: [SPARK-10337][SQL] fix hive views on non-hive-...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8990#issuecomment-146067831 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10337][SQL] fix hive views on non-hive-...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8990#issuecomment-146067781 [Test build #43310 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43310/console) for PR 8990 at commit

[GitHub] spark pull request: [SPARK-10941] [SQL] Refactor AggregateFunction...

2015-10-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/8973#discussion_r41348182 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala --- @@ -95,98 +92,192 @@ private[sql] case

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/8869#discussion_r41348491 --- Diff: R/pkg/R/stats.R --- @@ -0,0 +1,102 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/8869#discussion_r41348453 --- Diff: R/pkg/R/stats.R --- @@ -0,0 +1,102 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/8869#discussion_r41348531 --- Diff: R/pkg/R/stats.R --- @@ -0,0 +1,102 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request: [SPARK-10963] [Streaming] [Kafka] make KafkaCl...

2015-10-06 Thread koeninger
GitHub user koeninger opened a pull request: https://github.com/apache/spark/pull/9007 [SPARK-10963] [Streaming] [Kafka] make KafkaCluster public You can merge this pull request into a Git repository by running: $ git pull https://github.com/koeninger/spark-1 SPARK-10963

[GitHub] spark pull request: [SPARK-10963] [Streaming] [Kafka] make KafkaCl...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9007#issuecomment-146070202 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10963] [Streaming] [Kafka] make KafkaCl...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9007#issuecomment-146070196 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10963] [Streaming] [Kafka] make KafkaCl...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9007#issuecomment-146070719 [Test build #43315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43315/consoleFull) for PR 9007 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

2015-10-06 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146071744 @davies Yes. Currently as reported in the JIRA, using `x.asDict()["count"]` instead of `x.count` can be to work around this. But I think it would be better to avoid

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146071562 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146071597 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072217 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072238 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146072487 [Test build #43316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43316/consoleFull) for PR 8869 at commit

[GitHub] spark pull request: [SPARK-9478] [ml] Add class weights to Random ...

2015-10-06 Thread rotationsymmetry
GitHub user rotationsymmetry opened a pull request: https://github.com/apache/spark/pull/9008 [SPARK-9478] [ml] Add class weights to Random Forest This PR adds weight support to DecisionTreeClassifier DecisionTreeRegressor RandomForestClassifier

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146072680 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146072678 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10752][SPARKR] Implement corr() and cov...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8869#issuecomment-146072671 [Test build #43316 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43316/console) for PR 8869 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072983 [Test build #43317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43317/console) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072943 [Test build #43317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43317/consoleFull) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072985 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072984 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10963] [Streaming] [Kafka] make KafkaCl...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9007#issuecomment-146073351 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10963] [Streaming] [Kafka] make KafkaCl...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9007#issuecomment-146073306 [Test build #43315 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43315/console) for PR 9007 at commit

[GitHub] spark pull request: [SPARK-9478] [ml] Add class weights to Random ...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9008#issuecomment-146073233 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9478] [ml] Add class weights to Random ...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9008#issuecomment-146073241 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10064] [ML] Parallelize decision tree b...

2015-10-06 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/8246#issuecomment-146073328 OK thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...

2015-10-06 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/8967#discussion_r41349993 --- Diff: python/pyspark/mllib/clustering.py --- @@ -90,6 +90,14 @@ class KMeansModel(Saveable, Loader): ... rmtree(path) ... except

[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...

2015-10-06 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/8967#discussion_r41349992 --- Diff: python/pyspark/mllib/clustering.py --- @@ -90,6 +90,14 @@ class KMeansModel(Saveable, Loader): ... rmtree(path) ... except

[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...

2015-10-06 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/8967#discussion_r41349995 --- Diff: python/pyspark/mllib/clustering.py --- @@ -144,10 +152,16 @@ class KMeans(object): @classmethod def train(cls, rdd, k,

[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8967#issuecomment-146073431 [Test build #1849 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1849/consoleFull) for PR 8967 at commit

[GitHub] spark pull request: [SPARK-10963] [Streaming] [Kafka] make KafkaCl...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9007#issuecomment-146073349 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9478] [ml] Add class weights to Random ...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9008#issuecomment-146073764 [Test build #43318 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43318/consoleFull) for PR 9008 at commit

[GitHub] spark pull request: [SPARK-10887] [SQL] Build HashedRelation outsi...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8953#issuecomment-145943182 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10887] [SQL] Build HashedRelation outsi...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8953#issuecomment-145943213 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10887] [SQL] Build HashedRelation outsi...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8953#issuecomment-145943458 [Test build #43283 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43283/consoleFull) for PR 8953 at commit

[GitHub] spark pull request: [SPARK-10913][SPARKR] attach() function suppor...

2015-10-06 Thread felixcheung
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/8985#issuecomment-145951987 if we add attach shouldn't we add detach? https://stat.ethz.ch/R-manual/R-devel/library/base/html/attach.html Also, isn't `with` better for this so as to

[GitHub] spark pull request: [SPARK-10921] [YARN] Completely remove the use...

2015-10-06 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/8976#discussion_r41303229 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -147,23 +139,41 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: [SPARK-10810] [SPARK-10902] [SQL] Improve sess...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8909#issuecomment-145958655 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10913][SPARKR] attach() function suppor...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8997#issuecomment-145962605 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10913][SPARKR] attach() function suppor...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8997#issuecomment-145962536 [Test build #43286 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43286/console) for PR 8997 at commit

[GitHub] spark pull request: [SPARK-10917] [SQL] improve performance of com...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8971#issuecomment-145971061 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10887] [SQL] Build HashedRelation outsi...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8953#issuecomment-145974452 [Test build #43283 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43283/console) for PR 8953 at commit

[GitHub] spark pull request: [streaming] SPARK-10955. Disable dynamic alloc...

2015-10-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8998#issuecomment-145974489 This looks good. I personally prefer to fail fast loudly since this is a correctness issue. Many people don't pay attention to warnings these days, especially in

[GitHub] spark pull request: [SPARK-10885][Streaming]Display the failed out...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8950#issuecomment-145947026 [Test build #43278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43278/console) for PR 8950 at commit

[GitHub] spark pull request: [SPARK-10885][Streaming]Display the failed out...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8950#issuecomment-145947179 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10885][Streaming]Display the failed out...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8950#issuecomment-145947177 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8654][SQL] Fix Analysis exception when ...

2015-10-06 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/8983#issuecomment-145948447 @cloud-fan , please confirm my understanding of the code (fairly new to the codebase..:-) In the code we go through the entire in list and run evaluate the

[GitHub] spark pull request: [SPARK-10812] [yarn] Fix shutdown of token ren...

2015-10-06 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/8996 [SPARK-10812] [yarn] Fix shutdown of token renewer. A recent change to fix the referenced bug caused this exception in the `SparkContext.stop()` path: org.apache.spark.SparkException:

[GitHub] spark pull request: [SPARK-10812] [yarn] Fix shutdown of token ren...

2015-10-06 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8996#issuecomment-145949589 /cc @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-8848] [SQL] Refactors Parquet write pat...

2015-10-06 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/8988#issuecomment-145960253 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests

2015-10-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-145965472 Huh, looks like the last two failures were for the same PySpark test: ``` == FAIL

[GitHub] spark pull request: [streaming] SPARK-10955. Disable dynamic alloc...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8998#issuecomment-145966530 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [streaming] SPARK-10955. Disable dynamic alloc...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8998#issuecomment-145966515 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [streaming] SPARK-10955. Disable dynamic alloc...

2015-10-06 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/8998#issuecomment-145968060 Actually looking at this again, I think our only option is to log a message. It is possible that the `SparkContext` was already created and passed to us, in

[GitHub] spark pull request: [SPARK-8848] [SQL] Refactors Parquet write pat...

2015-10-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8988#issuecomment-145969709 [Test build #43291 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43291/consoleFull) for PR 8988 at commit

[GitHub] spark pull request: [SQL] [SPARK-5775] [SPARK-5508] Re-enable Hive...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8999#issuecomment-145971170 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10917] [SQL] improve performance of com...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8971#issuecomment-145971056 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SQL] [SPARK-5775] [SPARK-5508] Re-enable Hive...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8999#issuecomment-145971216 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [streaming] SPARK-10955. Disable dynamic alloc...

2015-10-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/8998#issuecomment-145972319 /cc @andrewor14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-8514] LU factorization on BlockMatrix

2015-10-06 Thread nilmeier
Github user nilmeier commented on the pull request: https://github.com/apache/spark/pull/8563#issuecomment-145943092 hello @dbtsai and thank you for the review comments. I will address your questions in the next few days if that is okay. One clarification: You indicated that

[GitHub] spark pull request: [SPARK-10917] [SQL] improve performance of com...

2015-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8971#issuecomment-145947938 Test PASSed. Refer to this link for build results (access rights to CI server needed):

  1   2   3   4   5   6   >