[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116763579 The PySpark ML tests passed, so this looks fine to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-8680][SQL] Slightly improve PropagateTy...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7087#issuecomment-116763775 [Test build #35991 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35991/console) for PR 7087 at commit

[GitHub] spark pull request: [SPARK-8680][SQL] Slightly improve PropagateTy...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7087#issuecomment-116763888 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-6707] [CORE][MESOS]: Mesos Scheduler sh...

2015-06-29 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/5563#issuecomment-116763524 @squito Thanks for reviewing and commenting, sorry I wasn't really clear I also don't think anything about Mesos should be merged without being tested with a real mesos

[GitHub] spark pull request: [SPARK-8066, SPARK-8067] [hive] Add support fo...

2015-06-29 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/7026#discussion_r33487305 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala --- @@ -32,13 +32,36 @@ package object client { // Hive 0.14 depends

[GitHub] spark pull request: [SPARK-8592] [CORE] CoarseGrainedExecutorBacke...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7077#issuecomment-116764897 [Test build #36004 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36004/consoleFull) for PR 7077 at commit

[GitHub] spark pull request: [SPARK-4176][WIP] Support decimal types with p...

2015-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/6796#discussion_r33487350 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/CatalystSchemaConverter.scala --- @@ -383,20 +386,14 @@ private[parquet] class

[GitHub] spark pull request: [SPARK-4176][WIP] Support decimal types with p...

2015-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/6796#discussion_r33487385 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/CatalystSchemaConverter.scala --- @@ -169,11 +169,12 @@ private[parquet] class

[GitHub] spark pull request: [SPARK-7708] [Core] [WIP] Fixes for Kryo closu...

2015-06-29 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/6361#issuecomment-116742527 @koertkuipers While it may be convenient for you that Scalding and Spark have a common serialization format when using chill/kryo, that can't be considered anything

[GitHub] spark pull request: [SPARK-7739][MLlib] Improve ChiSqSelector exam...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7029#issuecomment-116740165 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8592] [CORE] CoarseGrainedExecutorBacke...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7077#issuecomment-116744229 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8592] [CORE] CoarseGrainedExecutorBacke...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7077#issuecomment-116744260 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8554] Add the SparkR document files to ...

2015-06-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6947#issuecomment-116747028 @shivaram, I think the motivation here is to address local `dev/run-tests` failures that are caused by the presence of these documentation files: ```

[GitHub] spark pull request: [SPARK-8554] Add the SparkR document files to ...

2015-06-29 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/6947#issuecomment-116747612 Ah okay. In that case I'm fine with adding this. If the existing patch fixes the problem feel free to merge it --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-7810] [pyspark] solve python rdd socket...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6338#issuecomment-116751025 [Test build #35995 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35995/console) for PR 6338 at commit

[GitHub] spark pull request: [SPARK-7810] [pyspark] solve python rdd socket...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6338#issuecomment-116751053 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7088#issuecomment-116753470 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5155] [PySpark] [Streaming] Mqtt stream...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4229#issuecomment-116753493 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116759929 The spurious failure message from AmplabJenkins is a side-effect of some changes which skip the JVM tests for PRs that only touch certain PySpark modules. Ignore the

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-06-29 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5423#discussion_r33485446 --- Diff: yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryProvider.scala --- @@ -0,0 +1,1015 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...

2015-06-29 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/7088#issuecomment-116760825 Ah, that is after patch applied. O.K, I'll merge once the test finish successfully. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-4176][WIP] Support decimal types with p...

2015-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/6796#discussion_r33486747 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala --- @@ -43,16 +43,27 @@ private[parquet] object ParquetTypesConverter

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116763185 jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-8592] [CORE] CoarseGrainedExecutorBacke...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7077#issuecomment-116764561 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8592] [CORE] CoarseGrainedExecutorBacke...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7077#issuecomment-116764517 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4176][WIP] Support decimal types with p...

2015-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/6796#discussion_r33487239 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableSupport.scala --- @@ -369,9 +371,6 @@ private[parquet] class

[GitHub] spark pull request: [SPARK-8693][Project Infra]: profiles and goal...

2015-06-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7085#issuecomment-116740435 The script's lint checks looked fine and I manually verified that the message was logged properly, so I'm going to merge this into master. Thanks! --- If your

[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116740302 [Test build #35997 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35997/console) for PR 6830 at commit

[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116740290 [Test build #35997 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35997/consoleFull) for PR 6830 at commit

[GitHub] spark pull request: [SPARK-7739][MLlib] Improve ChiSqSelector exam...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7029#issuecomment-116740148 [Test build #35996 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35996/console) for PR 7029 at commit

[GitHub] spark pull request: [SPARK-8048] Partitionning of an RDD with 0 pa...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7020#issuecomment-116741869 [Test build #35998 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35998/consoleFull) for PR 7020 at commit

[GitHub] spark pull request: [SPARK-6797][SPARKR] Add support for YARN clus...

2015-06-29 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/6743#issuecomment-116748735 Thanks @sun-rui for the update. @davies can probably confirm but AFAIK the PySpark location is picked up at [1] by looking at the JAR file path / Spark home.

[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7088#issuecomment-116753571 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116757341 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116757293 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...

2015-06-29 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/7088#issuecomment-116758162 LGTM, but just in case, could you show a screen shot in the case of `totalExecutionTime == 0` ? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-7810] [pyspark] solve python rdd socket...

2015-06-29 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/6338#discussion_r33484881 --- Diff: python/pyspark/rdd.py --- @@ -121,15 +121,30 @@ def _parse_memory(s): def _load_from_socket(port, serializer): -sock =

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116759031 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...

2015-06-29 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/7088#issuecomment-116759370 could you show a screen shot in the case of totalExecutionTime == 0 ? I have already put a screenshot in the PR description. --- If your project is set up for

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116759076 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116760679 Thanks for the clarification! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-06-29 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/5423#issuecomment-116761481 Splitting up the patch? Would that help? Yes. Reviewing a 9k line patch is a lot of work, regardless of where the code is. People still need to review test

[GitHub] spark pull request: [SPARK-4176][WIP] Support decimal types with p...

2015-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/6796#discussion_r33486665 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala --- @@ -43,16 +43,27 @@ private[parquet] object ParquetTypesConverter

[GitHub] spark pull request: [SPARK-4176][WIP] Support decimal types with p...

2015-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/6796#discussion_r33486716 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala --- @@ -43,16 +43,27 @@ private[parquet] object ParquetTypesConverter

[GitHub] spark pull request: [SPARK-8592] [CORE] CoarseGrainedExecutorBacke...

2015-06-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7077#issuecomment-116763438 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8066, SPARK-8067] [hive] Add support fo...

2015-06-29 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/7026#discussion_r33487478 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala --- @@ -32,13 +32,36 @@ package object client { // Hive 0.14 depends

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116765543 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116766343 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116766385 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4176][WIP] Support decimal types with p...

2015-06-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/6796#issuecomment-116766442 Hi @rtreffer, - How is the compatibility mode intended to work? Settings are currently private, but I'd like to store Decimal(19), so is lifting the 18 limit

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-06-29 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5423#discussion_r33488335 --- Diff: yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryProvider.scala --- @@ -0,0 +1,1015 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-8581][SPARK-8584] Simplify checkpointin...

2015-06-29 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/6968#issuecomment-116769050 @tdas can you review? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8581][SPARK-8584] Simplify checkpointin...

2015-06-29 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/6968#issuecomment-116769034 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116769868 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116769791 [Test build #36005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36005/console) for PR 7058 at commit

[GitHub] spark pull request: [SPARK-8581][SPARK-8584] Simplify checkpointin...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6968#issuecomment-116770263 [Test build #36006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36006/consoleFull) for PR 6968 at commit

[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116770973 [Test build #36008 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36008/consoleFull) for PR 6830 at commit

[GitHub] spark pull request: [SPARK-8372] History server shows incorrect in...

2015-06-29 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/6827#issuecomment-116771070 @vanzin can you elaborate? Where is the offending line? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-6707] [CORE][MESOS]: Mesos Scheduler sh...

2015-06-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/5563#discussion_r33490036 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala --- @@ -59,6 +59,9 @@ private[spark] class

[GitHub] spark pull request: [SPARK-8681] fixed wrong ordering of columns i...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7060#issuecomment-116772124 [Test build #35993 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35993/console) for PR 7060 at commit

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116773193 Woohoo, looks like my hack worked :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-7810] [pyspark] solve python rdd socket...

2015-06-29 Thread AiHe
Github user AiHe commented on a diff in the pull request: https://github.com/apache/spark/pull/6338#discussion_r33490643 --- Diff: python/pyspark/rdd.py --- @@ -121,15 +121,30 @@ def _parse_memory(s): def _load_from_socket(port, serializer): -sock =

[GitHub] spark pull request: [SPARK-7810] [pyspark] solve python rdd socket...

2015-06-29 Thread AiHe
Github user AiHe commented on the pull request: https://github.com/apache/spark/pull/6338#issuecomment-116774008 When I'm working on python networking problem, I usually follow that example as a python standard usage. Then I just apply the underlying idea here. I'm wondering if I

[GitHub] spark pull request: [SPARK-8692][SQL] re-order the case statements...

2015-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/7073#discussion_r33492734 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnAccessor.scala --- @@ -71,44 +71,44 @@ private[sql] abstract class

[GitHub] spark pull request: [SPARK-3665][GraphX] Java API for GraphX

2015-06-29 Thread kdatta
Github user kdatta commented on the pull request: https://github.com/apache/spark/pull/3234#issuecomment-116766214 Hi Josh, Ankur, We have a dependence on this story for PyGraphX. We are close to completing the PyGraphX work and would like this code to be reviewed and

[GitHub] spark pull request: [SPARK-8704] [ML] [PySpark] Add additional met...

2015-06-29 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7086#issuecomment-116768373 It seems that the 'JMap[String, JList[Float]]' is not serializable. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116768318 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116768277 [Test build #35992 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35992/console) for PR 6830 at commit

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-06-29 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5423#discussion_r33488891 --- Diff: yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryProvider.scala --- @@ -0,0 +1,1015 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-4176][WIP] Support decimal types with p...

2015-06-29 Thread rtreffer
Github user rtreffer commented on the pull request: https://github.com/apache/spark/pull/6796#issuecomment-116769197 Hi @liancheng, thank you for the thorough review, will push a reworked version soon. Everything sounds reasonable :-) With private Settings I meant that I

[GitHub] spark pull request: [SPARK-6707] [CORE][MESOS]: Mesos Scheduler sh...

2015-06-29 Thread ankurcha
Github user ankurcha commented on the pull request: https://github.com/apache/spark/pull/5563#issuecomment-116769444 Thanks @tnachen @dragos - I did add some debug statements to make things explicit. Here is the output with `spark.mesos.constraints=zone:us-east-1a`:

[GitHub] spark pull request: [SPARK-8066, SPARK-8067] [hive] Add support fo...

2015-06-29 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/7026#discussion_r33489768 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala --- @@ -32,13 +32,36 @@ package object client { // Hive 0.14 depends

[GitHub] spark pull request: [SPARK-6707] [CORE][MESOS]: Mesos Scheduler sh...

2015-06-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/5563#discussion_r33490121 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala --- @@ -191,13 +194,29 @@ private[spark] class

[GitHub] spark pull request: [SPARK-8681] fixed wrong ordering of columns i...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7060#issuecomment-116772162 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-7735] [pyspark] Raise Exception on non-...

2015-06-29 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/6262#discussion_r33490447 --- Diff: python/pyspark/tests.py --- @@ -874,6 +874,15 @@ def test_sortByKey_uses_all_partitions_not_only_first_and_last(self): for size in

[GitHub] spark pull request: [SPARK-8704] [ML] [PySpark] Add additional met...

2015-06-29 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7086#issuecomment-116771077 I pushed in a hack that works, but now the Scala version looks ugly. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-8372] History server shows incorrect in...

2015-06-29 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/6827#issuecomment-116774379 OK, I'm going to revert this patch. @vanzin would you mind submitting a fix? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-06-29 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5423#discussion_r33491590 --- Diff: yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryProvider.scala --- @@ -0,0 +1,1015 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...

2015-06-29 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/7088#discussion_r33491939 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -570,6 +570,35 @@ private[ui] class StagePage(parent: StagesTab) extends

[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...

2015-06-29 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/7088#discussion_r33492287 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -570,6 +570,35 @@ private[ui] class StagePage(parent: StagesTab) extends

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116766793 I'm testing out a slightly silly hack to see if I can prevent the spurious AmplabJenkins errors: at the bottom of our Jenkins execute shell command, I'm adding an

[GitHub] spark pull request: [SPARK-4176][WIP] Support decimal types with p...

2015-06-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/6796#discussion_r33487973 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/CatalystSchemaConverter.scala --- @@ -383,20 +386,14 @@ private[parquet] class

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-06-29 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5423#discussion_r33488124 --- Diff: yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryProvider.scala --- @@ -0,0 +1,1015 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116767185 [Test build #36005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36005/consoleFull) for PR 7058 at commit

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-06-29 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5423#discussion_r33488601 --- Diff: yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryProvider.scala --- @@ -0,0 +1,1015 @@ +/* + * Licensed

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-06-29 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5423#discussion_r33488512 --- Diff: yarn/history/src/main/scala/org/apache/spark/deploy/history/yarn/YarnHistoryProvider.scala --- @@ -0,0 +1,1015 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-8066, SPARK-8067] [hive] Add support fo...

2015-06-29 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/7026#discussion_r33488555 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala --- @@ -32,13 +32,36 @@ package object client { // Hive 0.14 depends

[GitHub] spark pull request: [SPARK-8581][SPARK-8584] Simplify checkpointin...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6968#issuecomment-116769664 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8581][SPARK-8584] Simplify checkpointin...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6968#issuecomment-116769691 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116770692 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8704] [ML] [PySpark] Add additional met...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7086#issuecomment-116770659 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116770668 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8704] [ML] [PySpark] Add additional met...

2015-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7086#issuecomment-116770691 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6707] [CORE][MESOS]: Mesos Scheduler sh...

2015-06-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/5563#discussion_r33490259 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -86,10 +89,106 @@ private[mesos] trait

[GitHub] spark pull request: [SPARK-6707] [CORE][MESOS]: Mesos Scheduler sh...

2015-06-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/5563#discussion_r33490252 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -86,10 +89,106 @@ private[mesos] trait

[GitHub] spark pull request: [SPARK-6707] [CORE][MESOS]: Mesos Scheduler sh...

2015-06-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/5563#discussion_r33490244 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -86,10 +89,106 @@ private[mesos] trait

[GitHub] spark pull request: [SPARK-6707] [CORE][MESOS]: Mesos Scheduler sh...

2015-06-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/5563#discussion_r33490235 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -17,16 +17,19 @@ package

[GitHub] spark pull request: [SPARK-8704] [ML] [PySpark] Add additional met...

2015-06-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7086#issuecomment-116771747 [Test build #36007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36007/consoleFull) for PR 7086 at commit

[GitHub] spark pull request: [SPARK-8372] History server shows incorrect in...

2015-06-29 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/6827#issuecomment-116772492 @andrewor14 It's not a particular line, it's the whole patch. The patch ignores any application whose logs do not contain an application ID. No logs generated by Spark

[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...

2015-06-29 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/7088#discussion_r33491885 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -570,6 +570,35 @@ private[ui] class StagePage(parent: StagesTab) extends

<    1   2   3   4   5   6   7   8   9   10   >