[GitHub] spark pull request: [SPARK-15080][CORE] Break copyAndReset into co...

2016-05-05 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/12936 [SPARK-15080][CORE] Break copyAndReset into copy and reset ## What changes were proposed in this pull request? Break copyAndReset into two methods copy and reset instead of just one

[GitHub] spark pull request: [SPARK-928][CORE] Add support for Unsafe-based...

2016-05-05 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12913#issuecomment-217100211 @rxin figured out the problem, was using `var N` and its value was changed before the benchmark ran so each benchmark was running effectvely for `N=5000`, thus

[GitHub] spark pull request: [SPARK-15072][SQL][REPL][EXAMPLES] Remove Spar...

2016-05-05 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12851#issuecomment-217096326 @rxin @andrewor14 Comments Addressed, now all tests are passing 🎉 --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-928][CORE] Add support for Unsafe-based...

2016-05-04 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12913#issuecomment-217034618 @vectorijk The class Kryo Serializer is not wire-compatible, so IMO we should be ok. Either one uses Unsafe or safe based at a time. Comment from

[GitHub] spark pull request: [SPARK-928][CORE] Add support for Unsafe-based...

2016-05-04 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/12913 [SPARK-928][CORE] Add support for Unsafe-based serializer in Kryo ## What changes were proposed in this pull request? Now since we have migrated to Kryo-3.0.0 in https://issues.apache.org

[GitHub] spark pull request: [SPARK-15037][SQL][WIP] Use SparkSession inste...

2016-05-04 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/12907 [SPARK-15037][SQL][WIP] Use SparkSession instead of SQLContext in TestSuites ## What changes were proposed in this pull request? Use SparkSession instead of SQLContext in TestSuites

[GitHub] spark pull request: [SPARK-15072][SQL][REPL][EXAMPLES] Remove Spar...

2016-05-04 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12851#issuecomment-216924431 @rxin yupp failed tests are in sql hive tests, waiting for https://github.com/apache/spark/pull/12890 --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-15072][SQL][REPL][EXAMPLES] Remove Spar...

2016-05-03 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12851#issuecomment-216628798 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-15072][SQL][REPL][EXAMPLES] Remove Spar...

2016-05-03 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/12851#discussion_r61898180 --- Diff: sql/hive/src/test/resources/regression-test-SPARK-8489/Main.scala --- @@ -34,7 +34,7 @@ object Main { // scalastyle:off println

[GitHub] spark pull request: [SPARK-15087][CORE][SQL] Remove AccumulatorV2....

2016-05-03 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/12865#discussion_r61875706 --- Diff: core/src/main/scala/org/apache/spark/util/AccumulatorV2.scala --- @@ -126,23 +126,9 @@ abstract class AccumulatorV2[IN, OUT] extends

[GitHub] spark pull request: [SPARK-15087][CORE][SQL] Remove AccumulatorV2....

2016-05-03 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12865#issuecomment-216515443 @cloud-fan fixed 👍 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-15087][CORE][SQL] Remove AccumulatorV2....

2016-05-03 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/12865 [SPARK-15087][CORE][SQL] Remove AccumulatorV2.localValue and keep only value ## What changes were proposed in this pull request? Remove AccumulatorV2.localValue and keep only value

[GitHub] spark pull request: [SPARK-15082][CORE] Improve unit test coverage...

2016-05-03 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12862#issuecomment-216471037 @rxin hows it looking now ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-15082][CORE] Improve unit test coverage...

2016-05-03 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12862#issuecomment-216466924 @rxin fixed 👍 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Spark 15082

2016-05-03 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/12862 Spark 15082 ## What changes were proposed in this pull request? Added tests for ListAccumulator and LegacyAccumulatorWrapper, test for ListAccumulator is one similar to old Collection

[GitHub] spark pull request: [SPARK-15072][SQL][REPL][EXAMPLES] Remove Spar...

2016-05-02 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/12851#discussion_r61822413 --- Diff: sql/hive/src/test/resources/regression-test-SPARK-8489/Main.scala --- @@ -34,7 +34,7 @@ object Main { // scalastyle:off println

[GitHub] spark pull request: [SPARK-15072][SQL][REPL][EXAMPLES] Remove Spar...

2016-05-02 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/12851#discussion_r61821921 --- Diff: sql/hive/src/test/resources/regression-test-SPARK-8489/Main.scala --- @@ -34,7 +34,7 @@ object Main { // scalastyle:off println

[GitHub] spark pull request: [SPARK-15072][SQL][REPL][EXAMPLES] Remove Spar...

2016-05-02 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/12851 [SPARK-15072][SQL][REPL][EXAMPLES] Remove SparkSession.withHiveSupport ## What changes were proposed in this pull request? Removing the `withHiveSupport` method of `SparkSession`, instead

[GitHub] spark pull request: [SPARK-14422][SQL] Improve handling of optiona...

2016-05-02 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12846#issuecomment-216382056 Fixed 👍 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14422][SQL] Improve handling of optiona...

2016-05-02 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12846#issuecomment-216376049 @vanzin how's it looking now ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-14422][SQL] Improve handling of optiona...

2016-05-02 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/12846#discussion_r61808094 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -726,6 +726,14 @@ private[sql] class SQLConf extends Serializable

[GitHub] spark pull request: [SPARK-14422][SQL] Improve handling of optiona...

2016-05-02 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/12846#discussion_r61806604 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -726,6 +726,14 @@ private[sql] class SQLConf extends Serializable

[GitHub] spark pull request: [SPARK-14422][SQL] Improve handling of optiona...

2016-05-02 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/12846#discussion_r61803861 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -726,6 +726,14 @@ private[sql] class SQLConf extends Serializable

[GitHub] spark pull request: [SPARK-14422][SQL] Improve handling of optiona...

2016-05-02 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12846#issuecomment-216333816 cc: @vanzin @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14422][SQL] Improve handling of optiona...

2016-05-02 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/12846 [SPARK-14422][SQL] Improve handling of optional configs in SQLConf ## What changes were proposed in this pull request? Create a new API for handling Optional Configs in SQLConf. Right

[GitHub] spark pull request: [MINOR][DOCS] Fix type Information in Quick St...

2016-05-02 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12841#issuecomment-216322210 @srowen Can you merge this ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [MINOR][DOCS] Fix type Information in Quick St...

2016-05-02 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/12841#issuecomment-216227630 @srowen Yupp copied after running in spark-shell. https://cloud.githubusercontent.com/assets/1378755/14954856/45c281c2-1093-11e6-815b-c3beae3e05e6.png

[GitHub] spark pull request: Fix type Information in Quick Start and Progra...

2016-05-02 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/12841 Fix type Information in Quick Start and Programming Guide You can merge this pull request into a Git repository by running: $ git pull https://github.com/techaddict/spark improve_docs_1

[GitHub] spark pull request: [SPARK-9149][ML][Examples] Add an example of s...

2015-07-29 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/7697#discussion_r35740850 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/KMeansExample.scala --- @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-9149][ML][Examples] Add an example of s...

2015-07-29 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/7697#discussion_r35740783 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/KMeansExample.scala --- @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-1597: Add a version of reduceByKey that ...

2014-09-21 Thread techaddict
Github user techaddict closed the pull request at: https://github.com/apache/spark/pull/550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: Remove unnecessary Code from spark-shell.cmd

2014-07-31 Thread techaddict
Github user techaddict closed the pull request at: https://github.com/apache/spark/pull/666 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: SPARK-1869: 'spark-shell --help' fails if call...

2014-05-18 Thread techaddict
Github user techaddict closed the pull request at: https://github.com/apache/spark/pull/817 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: SPARK-1869: 'spark-shell --help' fails if call...

2014-05-18 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/817#issuecomment-43450550 ok --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: Remove unnecessary Code from spark-shell.cmd

2014-05-15 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/666#issuecomment-42392425 @pwendell can you review this ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-15 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/597#discussion_r12388904 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -88,7 +92,27 @@ object MovieLensALS { val

[GitHub] spark pull request: SPARK-1775: Unneeded lock in ShuffleMapTask.de...

2014-05-14 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/707#discussion_r12464716 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala --- @@ -58,15 +58,13 @@ private[spark] object ShuffleMapTask

[GitHub] spark pull request: SPARK-1775: Unneeded lock in ShuffleMapTask.de...

2014-05-14 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/707 SPARK-1775: Unneeded lock in ShuffleMapTask.deserializeInfo This was used in the past to have a cache of deserialized ShuffleMapTasks, but that's been removed, so there's no need for a lock

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-12 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/597#issuecomment-42459977 @mengxr changes done :smile: anything else or this good to merge ? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-11 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/597#discussion_r12389144 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -121,11 +157,14 @@ object MovieLensALS

[GitHub] spark pull request: Fix for SPARK-1758: failing test org.apache.sp...

2014-05-11 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/691#issuecomment-42520460 Why make pull requests for brach-1.0 and master both? I think #692 should be the only one. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-11 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/597#issuecomment-42404618 @mengxr Here are few results ``` implicitPref rank numInterations lambda - rmse true 10 20 1.0 - 0.5985187619423589 true

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-05-07 Thread techaddict
Github user techaddict closed the pull request at: https://github.com/apache/spark/pull/571 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-1637][HOTFIX] There are some Streaming ...

2014-05-07 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/673 [SPARK-1637][HOTFIX] There are some Streaming examples added after the PR #571 was last updated. This resulted in Compilation Errors. You can merge this pull request into a Git repository

[GitHub] spark pull request: [HOTFIX] SPARK-1637: There are some Streaming ...

2014-05-07 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/673#discussion_r12360143 --- Diff: examples/src/main/java/org/apache/spark/examples/streaming/JavaCustomReceiver.java --- @@ -15,7 +15,7 @@ * limitations under the License

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-07 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/597#discussion_r12362789 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -88,7 +92,11 @@ object MovieLensALS { val

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-07 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/597#discussion_r12363376 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -99,7 +107,12 @@ object MovieLensALS { val

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-07 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/597#discussion_r12362841 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -99,7 +107,12 @@ object MovieLensALS { val

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-07 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/597#discussion_r12362778 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -88,7 +92,11 @@ object MovieLensALS { val

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-07 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/597#discussion_r12363364 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -99,7 +107,12 @@ object MovieLensALS { val

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-06 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/597#issuecomment-42273835 @mengxr now good ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-05-06 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/571#issuecomment-42283416 @tdas ok I'll move streaming examples to org.apache.spark.streaming.examples.* as you suggest. Any other changes @mateiz --- If your project is set up for it, you

[GitHub] spark pull request: Remove unnecessary Code from spark-shell.cmd

2014-05-06 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/666 Remove unnecessary Code from spark-shell.cmd I don't see a reason to find path of bin directory. You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-05-06 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/571#issuecomment-42349536 @mateiz So this is good to merge. ? On May 7, 2014 1:13 AM, Matei Zaharia notificati...@github.com wrote: BTW the other point of this was to make them

[GitHub] spark pull request: SPARK-1710: spark-submit should print better e...

2014-05-04 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/630#discussion_r12260772 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -253,7 +254,12 @@ object SparkSubmit { val mainClass

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-02 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/597#issuecomment-42003703 Mapping rating in case of ImplicitPref to `{r=0 -- 0`, `r0 -- 1}`, `Rating(fields(0).toInt, fields(1).toInt, fields(2).toDouble)` to `Rating(fields(0).toInt

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-04-30 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/571#issuecomment-41765491 @mengxr @47ef86c392badc58052a0414115e49c2970b31eb looks good ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-04-30 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/597 SPARK-1668: Add implicit preference as an option to examples/MovieLensALS Add --implicitPrefs as an command-line option to the example app MovieLensALS under examples/ You can merge this pull

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-04-30 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/571#issuecomment-41835064 @mateiz ready for merging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1563 Add package-info.java and package.s...

2014-04-30 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/599#discussion_r12178082 --- Diff: bagel/src/main/scala/org/apache/spark/bagel/package-info.java --- @@ -0,0 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request: SPARK-1544 Add support for deep decision trees...

2014-04-30 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/475#discussion_r12178932 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/DecisionTree.scala --- @@ -72,7 +74,28 @@ class DecisionTree (private val strategy: Strategy

[GitHub] spark pull request: Update GradientDescentSuite.scala

2014-04-29 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/588#issuecomment-41685020 IMHO this will not compile if used like this `1 +: features`, Integers with DenseVector[Double]. Are you sure we want this change ? --- If your project is set up

[GitHub] spark pull request: Update GradientDescentSuite.scala

2014-04-29 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/588#discussion_r12102671 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/optimization/GradientDescentSuite.scala --- @@ -81,11 +81,11 @@ class GradientDescentSuite extends

[GitHub] spark pull request: Update GradientDescentSuite.scala

2014-04-29 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/588#discussion_r12102704 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/optimization/GradientDescentSuite.scala --- @@ -81,11 +81,11 @@ class GradientDescentSuite extends

[GitHub] spark pull request: Update GradientDescentSuite.scala

2014-04-29 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/588#issuecomment-41699346 @mengxr yupp that would be better. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-04-29 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/571#issuecomment-41701264 @mengxr IMO we should re-implement the necessary private functions in the examples, they are not too many just 2-3 methods. It will make using/modifying the examples

[GitHub] spark pull request: SPARK-1663. Corrections for several compile er...

2014-04-29 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/589#issuecomment-41703565 @srowen scala error is related to #550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-1663. Corrections for several compile er...

2014-04-29 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/589#issuecomment-41704140 While using these methods we have to specify the parameter type for the reduceFunc. https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache

[GitHub] spark pull request: Update GradientDescentSuite.scala

2014-04-29 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/588#issuecomment-41709531 @baishuo change it to `Vectors.dense(1.0 +: features.toArray)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-04-29 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/571#issuecomment-41758235 @mengxr so what changes should i make other than streaming one suggested by tdas. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-04-29 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/571#issuecomment-41761434 @mengxr what do you suggest how should we resolve the private[spark] problem ? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-04-27 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/571 SPARK-1637: Clean up examples for 1.0 1. Move all of them into subpackages of org.apache.spark.examples (right now some are in org.apache.spark.streaming.examples, for instance, and others

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-04-27 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/571#discussion_r12028250 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/RawTextHelper.scala --- @@ -22,10 +22,10 @@ import org.apache.spark.SparkContext

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-04-27 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/571#issuecomment-41492818 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-1637: Clean up examples for 1.0

2014-04-27 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/571#issuecomment-41506449 @mateiz We already have KMeans, LogisticRegressionWithSGD and other implementation examples as main functions in their respective objects. And can you whitelist me

[GitHub] spark pull request: SPARK-1597: Add a version of reduceByKey that ...

2014-04-26 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/550#discussion_r12023303 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -267,7 +267,7 @@ class ReceiverTracker(ssc

[GitHub] spark pull request: SPARK-1597: Add a version of reduceByKey that ...

2014-04-26 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/550#issuecomment-41462407 @rxin +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-1467: Make StorageLevel.apply() factory ...

2014-04-26 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/551#issuecomment-41485043 @mateiz done :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: spark.storage.memoryFraction indicates the max...

2014-04-26 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/567#issuecomment-41485102 @scwf don't worry about this test, its the hive test it takes 50+mins every time. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: SPARK-1597: Add a version of reduceByKey that ...

2014-04-25 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/550 SPARK-1597: Add a version of reduceByKey that takes the Partitioner as a... ... second argument Most of our shuffle methods can take a Partitioner or a number of partitions as a second

[GitHub] spark pull request: SPARK-1467: Make StorageLevel.apply() factory ...

2014-04-25 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/551 SPARK-1467: Make StorageLevel.apply() factory methods experimental We may want to evolve these in the future to add things like SSDs, so let's mark them as experimental for now. Long-term

[GitHub] spark pull request: SPARK-1557 Set permissions on event log files/...

2014-04-25 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/538#discussion_r12007808 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -51,10 +52,11 @@ private[spark] class EventLoggingListener

[GitHub] spark pull request: SPARK-1597: Add a version of reduceByKey that ...

2014-04-25 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/550#issuecomment-41455521 @mateiz I think this only applies with anon function's, thus isn't affecting either cogroup or groupByKey. --- If your project is set up for it, you can reply

[GitHub] spark pull request: SPARK-1467: Make StorageLevel.apply() factory ...

2014-04-25 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/551#issuecomment-41455586 @mateiz good to merge ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1467: Make StorageLevel.apply() factory ...

2014-04-25 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/551#issuecomment-41460577 Should i mark them dev then ? On Apr 26, 2014 10:55 AM, Patrick Wendell notificati...@github.com wrote: Thanks for this! I'd mark these as developer

[GitHub] spark pull request: Fix [SPARK-1078]: Replace lift-json with json4...

2014-04-24 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/536 Fix [SPARK-1078]: Replace lift-json with json4s-jackson. Remove the Unnecessary lift-json dependency from pom.xml You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: Fix Scala Style

2014-04-24 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/531#issuecomment-41317754 @rxin done :+1: :wink: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Getting absloute path instead of relative path...

2014-04-24 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/546#discussion_r11982803 --- Diff: core/src/main/scala/org/apache/spark/broadcast/HttpBroadcast.scala --- @@ -229,7 +229,7 @@ private[spark] object HttpBroadcast extends Logging

[GitHub] spark pull request: [Fix #79] Replace Breakable For Loops By While...

2014-04-23 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/503 [Fix #79] Replace Breakable For Loops By While Loops You can merge this pull request into a Git repository by running: $ git pull https://github.com/techaddict/spark fix-79 Alternatively

[GitHub] spark pull request: [Fix #79] Replace Breakable For Loops By While...

2014-04-23 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/503#issuecomment-41240253 @mateiz retest this please, I think failure is due to some other PR, as these changes have nothing to do with Streaming. --- If your project is set up for it, you

[GitHub] spark pull request: SPARK-1462: Examples of ML algorithms are usin...

2014-04-17 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/416#issuecomment-40685643 @pwendell :smile: i think this also closes SPARK-1464 ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: SPARK-1462: Examples of ML algorithms are usin...

2014-04-15 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/416 SPARK-1462: Examples of ML algorithms are using deprecated APIs This is a work in progress any comments are welcome. This will also fix SPARK-1464: Update MLLib Examples to Use Breeze. You can

[GitHub] spark pull request: SPARK-1462: Examples of ML algorithms are usin...

2014-04-15 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/416#issuecomment-40502741 @mengxr Ya i was thinking that too, as eventually we'll need function's like squaredDist(in KMeans Examples) implemented in mllib. --- If your project is set up

[GitHub] spark pull request: SPARK-1462: Examples of ML algorithms are usin...

2014-04-15 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/416#issuecomment-40506793 @srowen i think we need to implements some additional function's to `linalg.Vector` like squaredDist (supported by `util.Vector`) --- If your project is set up

[GitHub] spark pull request: SPARK-1462: Examples of ML algorithms are usin...

2014-04-15 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/416#issuecomment-40559039 @mengxr for now is this ok ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-1462: Examples of ML algorithms are usin...

2014-04-15 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/416#issuecomment-40560569 @mengxr fixed imports, for those not related to this PR, should i create a new PR ? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: SPARK-1469: Scheduler mode should accept lower...

2014-04-14 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/388#issuecomment-40446637 @pwendell IMHO the issue is resolved ? why not merge it ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: SPARK-1426: Make MLlib work with NumPy version...

2014-04-13 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/391#issuecomment-40301392 @mengxr yupp works with 1.4, 1.5 too :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1469: Scheduler mode should accept lower...

2014-04-13 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/388#issuecomment-40312774 @pwendell is there something else you were expecting ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: SPARK-1469: Scheduler mode should accept lower...

2014-04-12 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/388#issuecomment-40273462 @pwendell Done :+1: anything else ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

<    1   2   3   4   >