[GitHub] spark pull request: [SPARK-2850] [SPARK-2626] [mllib] MLlib stats ...

2014-08-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1878#discussion_r16338266 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/SampledRDDs.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-2862] histogram method fails on some ch...

2014-08-18 Thread nrchandan
Github user nrchandan commented on a diff in the pull request: https://github.com/apache/spark/pull/1787#discussion_r16338275 --- Diff: core/src/main/scala/org/apache/spark/rdd/DoubleRDDFunctions.scala --- @@ -96,6 +96,10 @@ class DoubleRDDFunctions(self: RDD[Double]) extends

[GitHub] spark pull request: [SPARK-2850] [SPARK-2626] [mllib] MLlib stats ...

2014-08-18 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1878#issuecomment-52455091 @jkbradley I tested the examples and found that `tree.py` is not included in `run-tests.py`. If we include it, it will throw errors due to `trainClassifier` needs at

[GitHub] spark pull request: [SPARK-3097][MLlib] Word2Vec performance impro...

2014-08-18 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1932#issuecomment-52456438 LGTM. Merged into master and branch-1.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2842][MLlib]Word2Vec documentation

2014-08-18 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2003#issuecomment-52456513 LGTM. Merged into master and branch-1.1. Thanks for working on the documentation!! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-08-18 Thread bgreeven
Github user bgreeven commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-52456503 Thanks for your feedback. I'll write some documentation, and also add some comments. I'll try with similar size data. The internal data structure of the weights

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16338954 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -159,6 +159,28 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16338944 --- Diff: sql/hive/src/main/scala/org/apache/spark/SparkHadoopWriter.scala --- @@ -93,6 +93,33 @@ private[hive] class SparkHiveHadoopWriter(

[GitHub] spark pull request: [SPARK-2850] [SPARK-2626] [mllib] MLlib stats ...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1878#issuecomment-52456910 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18738/consoleFull) for PR 1878 at commit

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16338953 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -159,6 +159,28 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [MLlib] Remove transform(dataset: RDD[String])...

2014-08-18 Thread Ishiihara
GitHub user Ishiihara opened a pull request: https://github.com/apache/spark/pull/2010 [MLlib] Remove transform(dataset: RDD[String]) from Word2Vec public API @mengxr Remove transform(dataset: RDD[String]) from public API. You can merge this pull request into a Git repository

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16338992 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -178,6 +200,12 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16339033 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -204,12 +235,81 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16339036 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -204,12 +235,81 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16339176 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -204,12 +235,81 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16339200 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -204,12 +235,81 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-1853] Show Streaming application code c...

2014-08-18 Thread mubarak
Github user mubarak commented on a diff in the pull request: https://github.com/apache/spark/pull/1723#discussion_r16339260 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -813,7 +816,10 @@ private[spark] object Utils extends Logging { * A regular

[GitHub] spark pull request: [SPARK-3063][SQL] ExistingRdd should convert M...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1963#issuecomment-52457958 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18739/consoleFull) for PR 1963 at commit

[GitHub] spark pull request: [MLlib] Remove transform(dataset: RDD[String])...

2014-08-18 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2010#issuecomment-52458146 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [MLlib] Remove transform(dataset: RDD[String])...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2010#issuecomment-52458413 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18744/consoleFull) for PR 2010 at commit

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16339566 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -231,14 +327,26 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16339580 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -231,14 +327,26 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-2842][MLlib]Word2Vec documentation

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2003#issuecomment-52458933 **Tests timed out** after a configured wait of `120m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1919#issuecomment-52459171 @yhuai It would be nice if you can have a look at this PR as you're the expert here :) @baishuo You can refer to `sql/README.md` for details about setting up

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16340005 --- Diff: sql/hive/src/main/scala/org/apache/spark/SparkHadoopWriter.scala --- @@ -93,6 +93,33 @@ private[hive] class SparkHiveHadoopWriter(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16340258 --- Diff: sql/hive/src/main/scala/org/apache/spark/SparkHadoopWriter.scala --- @@ -93,6 +93,33 @@ private[hive] class SparkHiveHadoopWriter(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16340321 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -159,6 +162,36 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16340359 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -201,15 +270,78 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16340415 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -159,6 +162,36 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16340505 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -159,6 +162,36 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [MLlib] Remove transform(dataset: RDD[String])...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2010#issuecomment-52462145 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18744/consoleFull) for PR 2010 at commit

[GitHub] spark pull request: [MLlib] Remove transform(dataset: RDD[String])...

2014-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2010 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2907] [MLlib] Use mutable.HashMap to re...

2014-08-18 Thread Ishiihara
Github user Ishiihara closed the pull request at: https://github.com/apache/spark/pull/1871 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2862] histogram method fails on some ch...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1787#issuecomment-52463719 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18745/consoleFull) for PR 1787 at commit

[GitHub] spark pull request: remove MaxPermSize option for jvm 1.8

2014-08-18 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/2011 remove MaxPermSize option for jvm 1.8 In JVM 1.8.0, MaxPermSize is no longer supported. In spark `stderr` output, there would be a line of Java HotSpot(TM) 64-Bit Server VM

[GitHub] spark pull request: remove MaxPermSize option for jvm 1.8

2014-08-18 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/2011#issuecomment-52467109 @aarondav I have modified codes according to your comments, and since I did a rebase, I have to start a new PR... --- If your project is set up for it, you can

[GitHub] spark pull request: remove MaxPermSize option for jvm 1.8

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2011#issuecomment-52467263 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18746/consoleFull) for PR 2011 at commit

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16343293 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -191,7 +257,10 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-2862] histogram method fails on some ch...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1787#issuecomment-52468388 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18745/consoleFull) for PR 1787 at commit

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16343341 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -201,17 +270,79 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16343351 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -201,17 +270,79 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16343370 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -201,17 +270,79 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16343382 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -201,17 +270,79 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16343397 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -201,17 +270,79 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16343411 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -201,17 +270,79 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16343705 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -201,17 +270,79 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16343814 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -201,17 +270,79 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16344256 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -191,7 +257,10 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: remove MaxPermSize option for jvm 1.8

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2011#issuecomment-52472156 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18746/consoleFull) for PR 2011 at commit

[GitHub] spark pull request: [SPARK-2927][SQL] Add a conf to configure if w...

2014-08-18 Thread chutium
Github user chutium commented on a diff in the pull request: https://github.com/apache/spark/pull/1855#discussion_r16345194 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala --- @@ -403,7 +406,10 @@ private[parquet] object ParquetTypesConverter

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16345316 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -159,6 +162,36 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16345513 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -191,7 +257,10 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: Move a bracket in validateSettings of SparkCon...

2014-08-18 Thread hzw19900416
GitHub user hzw19900416 opened a pull request: https://github.com/apache/spark/pull/2012 Move a bracket in validateSettings of SparkConf Move a bracket in validateSettings of SparkConf You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: Move a bracket in validateSettings of SparkCon...

2014-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2012#issuecomment-52474725 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16346630 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -191,7 +257,10 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16346660 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -191,7 +257,10 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16347194 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -191,7 +257,10 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: Delete unused Travis CI config

2014-08-18 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/2013 Delete unused Travis CI config Per @rxin at https://github.com/apache/spark/pull/1987#issuecomment-52405072 , Travis CI config is not used and can be deleted. I didn't see anyone had picked that

[GitHub] spark pull request: [BUILD] Delete unused Travis CI config

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2013#issuecomment-52485853 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18747/consoleFull) for PR 2013 at commit

[GitHub] spark pull request: SPARK-3069 [DOCS] Build instructions in README...

2014-08-18 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/2014 SPARK-3069 [DOCS] Build instructions in README are outdated Here's my crack at Bertrand's suggestion. The Github `README.md` contains build info that's outdated. It should just point to the current

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-08-18 Thread akopich
Github user akopich commented on a diff in the pull request: https://github.com/apache/spark/pull/1269#discussion_r16351508 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/topicmodeling/utils/serialization/TObjectIntHashMapSerializer.scala --- @@ -0,0 +1,51 @@

[GitHub] spark pull request: [SPARK-2873] [SQL] using ExternalAppendOnlyMap...

2014-08-18 Thread guowei2
Github user guowei2 commented on the pull request: https://github.com/apache/spark/pull/1822#issuecomment-52489381 i'm very sorry. i just rebase my brach to spark/master. what should i do to fix this . --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16351653 --- Diff: sql/hive/src/main/scala/org/apache/spark/SparkHadoopWriter.scala --- @@ -93,6 +93,33 @@ private[hive] class SparkHiveHadoopWriter( null)

[GitHub] spark pull request: [SPARK-2873] [SQL] using ExternalAppendOnlyMap...

2014-08-18 Thread guowei2
Github user guowei2 commented on the pull request: https://github.com/apache/spark/pull/1822#issuecomment-52490972 may i close this PR and create a new PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [BUILD] Delete unused Travis CI config

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2013#issuecomment-52491425 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18747/consoleFull) for PR 2013 at commit

[GitHub] spark pull request: [SPARK-2873] [SQL] using ExternalAppendOnlyMap...

2014-08-18 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1822#issuecomment-52491562 Try this: `git commit -m Big-ass commit --allow-empty` `git rebase -i master`, `git push origin sql-memory-patch -f ` --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-2974] [SPARK-2975] Fix two bugs related...

2014-08-18 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/2002#discussion_r16353011 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -451,10 +451,56 @@ private[spark] object Utils extends Logging { /** *

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-52493510 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18749/consoleFull) for PR 1269 at commit

[GitHub] spark pull request: [SPARK-1720] Add the value of LD_LIBRARY_PATH ...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-52495578 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18750/consoleFull) for PR 1031 at commit

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-52500225 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18749/consoleFull) for PR 1269 at commit

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-08-18 Thread akopich
Github user akopich commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-52500502 @mengxr I have tried my best to make interfaces minimal and comment them in understandable way. Please, could you mention what exactly is unclear? --- If your

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-08-18 Thread akopich
Github user akopich commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-52499944 @chazchandler, As far as I can see, you run RobustPLSA. Behavious like this may take place if every word from a document is explained by background or noise.

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/1919#discussion_r16358186 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -201,17 +270,79 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-1720] Add the value of LD_LIBRARY_PATH ...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-52503551 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18750/consoleFull) for PR 1031 at commit

[GitHub] spark pull request: [mllib] DecisionTree: treeAggregate + Python e...

2014-08-18 Thread jkbradley
GitHub user jkbradley opened a pull request: https://github.com/apache/spark/pull/2015 [mllib] DecisionTree: treeAggregate + Python example bug fix Small DecisionTree updates: * Changed main DecisionTree aggregate to treeAggregate. * Fixed bug in python example

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-52507178 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18753/consoleFull) for PR 1269 at commit

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-52509222 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18751/consoleFull) for PR 1269 at commit

[GitHub] spark pull request: [SPARK-3007][SQL]Add Dynamic Partition suppo...

2014-08-18 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/1919#issuecomment-52509183 @baishuo Thank you for working on it. I have three general comments. 1. Hive has a lots of confs that are used to influence how semantic analyzer works.

[GitHub] spark pull request: [SPARK-3101] Flag variable in ApplicationMaste...

2014-08-18 Thread sarutak
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/2016 [SPARK-3101] Flag variable in ApplicationMaster should be declared as volatile You can merge this pull request into a Git repository by running: $ git pull https://github.com/sarutak/spark

[GitHub] spark pull request: [SPARK-3101] Flag variable in ApplicationMaste...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2016#issuecomment-52510886 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18754/consoleFull) for PR 2016 at commit

[GitHub] spark pull request: [SPARK-2850] [SPARK-2626] [mllib] MLlib stats ...

2014-08-18 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/1878#issuecomment-52512515 @mengxr Thanks! I'll send the tree fixes in the other PR I sent just now on treeAggregate(), and I will do the keyCount fix in this PR. --- If your project is set up

[GitHub] spark pull request: [mllib] DecisionTree: treeAggregate + Python e...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2015#issuecomment-52513939 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18755/consoleFull) for PR 2015 at commit

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-52515294 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18753/consoleFull) for PR 1269 at commit

[GitHub] spark pull request: [mllib] DecisionTree: treeAggregate + Python e...

2014-08-18 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/2015#issuecomment-52513395 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-08-18 Thread chazchandler
Github user chazchandler commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-52515801 Is there a straightforward way to get the `topics` themselves back out of the `infer` step? They seem to be [effectively private inside of

[GitHub] spark pull request: [SPARK-2974] [SPARK-2975] Fix two bugs related...

2014-08-18 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/2002#discussion_r16363882 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -451,10 +451,56 @@ private[spark] object Utils extends Logging { /** *

[GitHub] spark pull request: [SPARK-3012] Standardized Distance Functions b...

2014-08-18 Thread erikerlandson
Github user erikerlandson commented on the pull request: https://github.com/apache/spark/pull/1964#issuecomment-52516267 I'm wondering if it might be simpler and more idiomatic to just define distance measure directly as any subclass of Function2, like: trait DistanceMeasure

[GitHub] spark pull request: [SPARK-2974] [SPARK-2975] Fix two bugs related...

2014-08-18 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/2002#discussion_r16364099 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -451,10 +451,56 @@ private[spark] object Utils extends Logging { /** *

[GitHub] spark pull request: SPARK-3093 : masterLock in Worker is no longer...

2014-08-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/2008#issuecomment-52518124 Thanks for the patch, based on the brief discussion

[GitHub] spark pull request: [SPARK-3101] Flag variable in ApplicationMaste...

2014-08-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2016#issuecomment-52519179 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18754/consoleFull) for PR 2016 at commit

[GitHub] spark pull request: [SPARK-2974] [SPARK-2975] Fix two bugs related...

2014-08-18 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/2002#discussion_r16365281 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -451,10 +451,56 @@ private[spark] object Utils extends Logging { /** *

[GitHub] spark pull request: [SPARK-2974] [SPARK-2975] Fix two bugs related...

2014-08-18 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/2002#discussion_r16365389 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -451,10 +451,56 @@ private[spark] object Utils extends Logging { /** *

[GitHub] spark pull request: [SPARK-2848] Shade Guava in uber-jars.

2014-08-18 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/1813#issuecomment-52520416 So, any more feedback on this patch? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2169] Don't copy appName / basePath eve...

2014-08-18 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/1252#issuecomment-52520495 Hi, could we get this merged if there's no more feedback? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-2718] [yarn] Handle quotes and other ch...

2014-08-18 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/1724#issuecomment-52520456 Ping. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-2862] histogram method fails on some ch...

2014-08-18 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1787#issuecomment-52520863 LGTM. Merged into master and branch-1.1. Thanks!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [mllib] DecisionTree: treeAggregate + Python e...

2014-08-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2015#discussion_r16366057 --- Diff: examples/src/main/python/mllib/decision_tree_runner.py --- @@ -124,7 +124,9 @@ def usage(): (reindexedData, origToNewLabels) =

[GitHub] spark pull request: [SPARK-2862] histogram method fails on some ch...

2014-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1787 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2718] [yarn] Handle quotes and other ch...

2014-08-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1724#issuecomment-52521078 @vanzin I'm going to quickly test this on a yarn cluster first, and then I'll merge it. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [mllib] DecisionTree: treeAggregate + Python e...

2014-08-18 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2015#issuecomment-52521130 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

  1   2   3   4   >