[GitHub] spark pull request: [SPARK-7662] [SQL] Resolve correct names for g...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6178#issuecomment-102941559 [Test build #32975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32975/consoleFull) for PR 6178 at commit [`5c3f2c5`](https://gith

[GitHub] spark pull request: SPARK-7063 when lz4 compression is used, it ca...

2015-05-17 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5641#issuecomment-102940913 (That's fine -- you can also just rebase and force-push the existing branch/PR. The advantage there is that it retains the discussion in a single PR. But no big deal here

[GitHub] spark pull request: Spark-7063

2015-05-17 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/6226#issuecomment-102941051 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark pull request: [SPARK-7662] [SQL] Resolve correct names for g...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6178#issuecomment-102940811 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7662] [SQL] Resolve correct names for g...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6178#issuecomment-102940886 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6980] [CORE] [WIP] Akka timeout excepti...

2015-05-17 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/6205#discussion_r30484698 --- Diff: core/src/main/scala/org/apache/spark/util/RpcUtils.scala --- @@ -47,14 +46,24 @@ object RpcUtils { } /** Returns the default

[GitHub] spark pull request: [SPARK-7662] [SQL] Resolve correct names for g...

2015-05-17 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/6178#discussion_r30484488 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -576,6 +576,13 @@ class Analyzer( /** Extr

[GitHub] spark pull request: [SPARK-7322] [SQL] [WIP] Support Window Functi...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6104#issuecomment-102937501 [Test build #32974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32974/consoleFull) for PR 6104 at commit [`3dc8d09`](https://gith

[GitHub] spark pull request: [SPARK-7251] Perform sequential scan when iter...

2015-05-17 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/6159#discussion_r30484136 --- Diff: unsafe/src/test/java/org/apache/spark/unsafe/map/AbstractBytesToBytesMapSuite.java --- @@ -197,6 +216,74 @@ public void iteratorTest() throws Excep

[GitHub] spark pull request: [SPARK-7251] Perform sequential scan when iter...

2015-05-17 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/6159#issuecomment-102937301 How about make BytesToBytesMap implement `Iterable` so that it can be used in for-each? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-7322] [SQL] [WIP] Support Window Functi...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6104#issuecomment-102937320 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7322] [SQL] [WIP] Support Window Functi...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6104#issuecomment-102937305 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7681][MLlib] Add SparseVector support f...

2015-05-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/6209#discussion_r30483846 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala --- @@ -473,44 +473,161 @@ private[spark] object BLAS extends Serializable with Logging

[GitHub] spark pull request: [SPARK-7251] Perform sequential scan when iter...

2015-05-17 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/6159#discussion_r30483774 --- Diff: unsafe/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -162,6 +169,55 @@ public BytesToBytesMap( */ public int

[GitHub] spark pull request: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6221#issuecomment-102933201 [Test build #32972 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32972/consoleFull) for PR 6221 at commit [`e3f83fe`](https://gith

[GitHub] spark pull request: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6221#issuecomment-102932463 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7627][SPARK-7472] DAG visualization: st...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6171#issuecomment-102932556 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6221#issuecomment-102932557 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7627][SPARK-7472] DAG visualization: st...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6171#issuecomment-102932734 [Test build #32973 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32973/consoleFull) for PR 6171 at commit [`f261797`](https://gith

[GitHub] spark pull request: [SPARK-7627][SPARK-7472] DAG visualization: st...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6171#issuecomment-102932473 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7251] Perform sequential scan when iter...

2015-05-17 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/6159#issuecomment-102932239 @JoshRosen could you explain what will happen if `oldCapacity == (1 << 30)` in https://github.com/JoshRosen/spark/blob/SPARK-7251/unsafe/src/main/java/org/apache/spark/u

[GitHub] spark pull request: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-17 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/6221#issuecomment-102932101 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SPARK-7681][MLlib] Add SparseVector support f...

2015-05-17 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/6209#discussion_r30483433 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/BLASSuite.scala --- @@ -257,32 +257,60 @@ class BLASSuite extends FunSuite { new DenseM

[GitHub] spark pull request: [SPARK-7681][MLlib] Add SparseVector support f...

2015-05-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/6209#discussion_r30483386 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/BLASSuite.scala --- @@ -257,32 +257,60 @@ class BLASSuite extends FunSuite { new DenseM

[GitHub] spark pull request: [SPARK-6530][ML] Add chi-square selector for m...

2015-05-17 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/5742#issuecomment-102931364 @jkbradley Sure, I will check the new PR soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-7380][MLLIB] pipeline stages should be ...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6088#issuecomment-102930279 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32

[GitHub] spark pull request: [SPARK-7380][MLLIB] pipeline stages should be ...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6088#issuecomment-102930246 [Test build #32966 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32966/consoleFull) for PR 6088 at commit [`413c463`](https://git

[GitHub] spark pull request: [SPARK-7380][MLLIB] pipeline stages should be ...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6088#issuecomment-102930274 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102929931 [Test build #32971 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32971/consoleFull) for PR 6220 at commit [`608cdc9`](https://gith

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102929765 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102929792 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102929501 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102928953 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102928954 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102928946 [Test build #32968 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32968/consoleFull) for PR 6220 at commit [`608cdc9`](https://git

[GitHub] spark pull request: [SPARK-7150] add range() api

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6081#issuecomment-102928597 [Test build #32970 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32970/consoleFull) for PR 6081 at commit [`cbf5200`](https://gith

[GitHub] spark pull request: [SPARK-7150] add range() api

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6081#issuecomment-102928445 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7150] add range() api

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6081#issuecomment-102928435 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7322] [SQL] [WIP] Support Window Functi...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6104#issuecomment-102928275 [Test build #32969 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32969/consoleFull) for PR 6104 at commit [`4553993`](https://gith

[GitHub] spark pull request: [SPARK-6980] [CORE] [WIP] Akka timeout excepti...

2015-05-17 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/6205#issuecomment-102928153 @hardmettle are you able to checkout my branch, then maybe you could share any changes/additions as a pull-request? --- If your project is set up for it, you can re

[GitHub] spark pull request: [SPARK-7322] [SQL] [WIP] Support Window Functi...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6104#issuecomment-102928135 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7322] [SQL] [WIP] Support Window Functi...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6104#issuecomment-102928142 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7673] [SQL] WIP: Moves file status cach...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6225#issuecomment-102927851 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-7673] [SQL] WIP: Moves file status cach...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6225#issuecomment-102927852 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32

[GitHub] spark pull request: [SPARK-7673] [SQL] WIP: Moves file status cach...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6225#issuecomment-102927845 [Test build #32964 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32964/consoleFull) for PR 6225 at commit [`ba41250`](https://git

[GitHub] spark pull request: [SPARK-6806] [SparkR] [Docs] Fill in SparkR ex...

2015-05-17 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/5442#discussion_r30482155 --- Diff: docs/sql-programming-guide.md --- @@ -491,6 +573,37 @@ for teenName in teenNames.collect(): + + --- End diff --

[GitHub] spark pull request: [SPARK-7627][SPARK-7472] DAG visualization: st...

2015-05-17 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/6171#issuecomment-102923806 Cool, this looks pretty good. Maybe I'd make the border on skipped stages slightly darker since the red is darker than blue on the other ones, but overall it's fine. --

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102923624 R tests LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-7698] [WIP] Cache and reuse buffers in ...

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6227#issuecomment-102923557 Leaving the [WIP] tag on for now while we discuss a few different design decisions. --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-7698] [WIP] Cache and reuse buffers in ...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6227#issuecomment-102923072 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32

[GitHub] spark pull request: [SPARK-7698] [WIP] Cache and reuse buffers in ...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6227#issuecomment-102923064 [Test build #32963 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32963/consoleFull) for PR 6227 at commit [`b154e86`](https://git

[GitHub] spark pull request: [SPARK-7698] [WIP] Cache and reuse buffers in ...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6227#issuecomment-102923071 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6980] [CORE] [WIP] Akka timeout excepti...

2015-05-17 Thread hardmettle
Github user hardmettle commented on the pull request: https://github.com/apache/spark/pull/6205#issuecomment-102922877 @BryanCutler can I get write access to your PR repo ? I want to merge my changes , how do I do it ? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102921594 [Test build #32968 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32968/consoleFull) for PR 6220 at commit [`608cdc9`](https://gith

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102921063 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102921117 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7689] Deprecate spark.cleaner.ttl

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6220#issuecomment-102920879 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7691] [WIP] Refactor CatalystTypeConver...

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6222#issuecomment-102920647 Based on the last set of test failures, it looks like there's another other corner-case where the actual data types don't match the declared ones. **JavaHashin

[GitHub] spark pull request: [SPARK-7698] [WIP] Cache and reuse buffers in ...

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6227#issuecomment-102920246 For https://gist.github.com/680ee530655941defcb2, this patch gives a roughly 3x speedup. --- If your project is set up for it, you can reply to this email and have yo

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102919872 [Test build #32967 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32967/consoleFull) for PR 6218 at commit [`5946da9`](https://gith

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102919830 I've updated the code and PR description for the string cast. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6221#issuecomment-102919795 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6221#issuecomment-102919796 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102919772 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102919768 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7380][MLLIB] pipeline stages should be ...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6088#issuecomment-102917618 [Test build #32966 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32966/consoleFull) for PR 6088 at commit [`413c463`](https://gith

[GitHub] spark pull request: [SPARK-7380][MLLIB] pipeline stages should be ...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6088#issuecomment-102917503 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7380][MLLIB] pipeline stages should be ...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6088#issuecomment-102917464 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: Spark-7063

2015-05-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6226#issuecomment-102917278 Can you update the pull request title to make it more informative? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark pull request: [SPARK-7673] [SQL] WIP: Moves file status cach...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6225#issuecomment-102917177 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32

[GitHub] spark pull request: [SPARK-7673] [SQL] WIP: Moves file status cach...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6225#issuecomment-102917174 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-7673] [SQL] WIP: Moves file status cach...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6225#issuecomment-102917167 [Test build #32962 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32962/consoleFull) for PR 6225 at commit [`3d278f7`](https://git

[GitHub] spark pull request: [SPARK-7380][MLLIB] pipeline stages should be ...

2015-05-17 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/6088#issuecomment-102917145 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SPARK-7694] [MLlib] Use getOrElse for getting...

2015-05-17 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/6224#issuecomment-102917086 LGTM. Merged into master and branch-1.4. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-6980] [CORE] [WIP] Akka timeout excepti...

2015-05-17 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/6205#discussion_r30480924 --- Diff: core/src/main/scala/org/apache/spark/rpc/akka/AkkaRpcEnv.scala --- @@ -212,7 +211,7 @@ private[spark] class AkkaRpcEnv private[akka] (

[GitHub] spark pull request: [SPARK-7694] [MLlib] Use getOrElse for getting...

2015-05-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6224 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6221#issuecomment-102917028 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6221#issuecomment-102917026 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6657] [PYSPARK] Fix doc warnings

2015-05-17 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/6221#issuecomment-102916941 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102916523 Whoops, looks like I overlooked the Scaladoc comment: ``` * This function is meant for exploratory data analysis, as we make no guarantee about the

[GitHub] spark pull request: [SPARK-7631] [SQL] treenode argString should n...

2015-05-17 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/6144#issuecomment-102915457 /cc @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark pull request: [SPARK-6980] [CORE] [WIP] Akka timeout excepti...

2015-05-17 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/6205#discussion_r30480490 --- Diff: core/src/main/scala/org/apache/spark/rpc/RpcEnv.scala --- @@ -94,7 +97,7 @@ private[spark] abstract class RpcEnv(conf: SparkConf) { * Ret

[GitHub] spark pull request: [SPARK-7389] [core]Tachyon integration improve...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5908#issuecomment-102915182 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-7389] [core]Tachyon integration improve...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5908#issuecomment-102915185 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32

[GitHub] spark pull request: [SPARK-7389] [core]Tachyon integration improve...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5908#issuecomment-102915177 [Test build #32961 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32961/consoleFull) for PR 5908 at commit [`137f3d2`](https://git

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102915137 describe should only be used for interactive data analysis -- i.e. no users should depend on its output. We should perhaps update documentation to include that. --- If yo

[GitHub] spark pull request: [SPARK-7673] [SQL] WIP: Moves file status cach...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6225#issuecomment-102914987 [Test build #32964 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32964/consoleFull) for PR 6225 at commit [`ba41250`](https://gith

[GitHub] spark pull request: [SPARK-7673] [SQL] WIP: Moves file status cach...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6225#issuecomment-102914544 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102914723 My concern was users who might be relying on the existing defined behavior of this returning all double columns. Basically, it seems like aggregating over all numeric

[GitHub] spark pull request: [SPARK-7673] [SQL] WIP: Moves file status cach...

2015-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6225#issuecomment-102914507 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102913682 It's not that big of a deal to optimize this. Basically describe is only used for interactive data exploration for viewing results, so any data type that doesn't crash the

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102913016 Maybe we could use some sort of least-upper-bound to determine which cast to perform. For instance, if we have all numeric columns, then perform a Double cast and dec

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102912011 If we actually support running describe on non-numeric columns, then stringtype is the only option. --- If your project is set up for it, you can reply to this email and h

[GitHub] spark pull request: [SPARK-7693][Core] Remove "import scala.concur...

2015-05-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6223 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-7693][Core] Remove "import scala.concur...

2015-05-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6223#issuecomment-102910250 LGTM. Merging thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102910198 Yeah, it seems to because it looks like we don't actually enforce types: ```scala val describeNonNumericColumn = describeTestData.describe("name") `

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102909093 Does describe work on non-numeric columns? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [SPARK-6707] [CORE][MESOS]: Mesos Scheduler sh...

2015-05-17 Thread ankurcha
Github user ankurcha commented on a diff in the pull request: https://github.com/apache/spark/pull/5563#discussion_r30479950 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -86,10 +90,101 @@ private[mesos] trait MesosSchedule

[GitHub] spark pull request: [SPARK-7687] [SQL] DataFrame.describe() should...

2015-05-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6218#issuecomment-102908528 @rxin, it looks like this test is failing because the R test expects the `min` of a string-valued column to return a string: ```r test_that("describe() on

[GitHub] spark pull request: [SPARK-7698] [WIP] Cache and reuse buffers in ...

2015-05-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6227#issuecomment-102907369 [Test build #32963 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32963/consoleFull) for PR 6227 at commit [`b154e86`](https://gith

  1   2   3   4   >