[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15438 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66762/ Test PASSed. ---

[GitHub] spark issue #11336: [SPARK-9325][SPARK-R] collect() head() and show() for Co...

2016-10-11 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/11336 I know why tests fail - please see my comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15438 **[Test build #66762 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66762/consoleFull)** for PR 15438 at commit

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82908836 --- Diff: R/pkg/R/functions.R --- @@ -3026,7 +3048,11 @@ setMethod("translate", setMethod("unix_timestamp", signature(x = "missing", format =

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82908811 --- Diff: R/pkg/R/functions.R --- @@ -2865,7 +2882,11 @@ setMethod("rand", signature(seed = "numeric"), setMethod("randn", signature(seed =

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82908799 --- Diff: R/pkg/R/functions.R --- @@ -2847,7 +2860,11 @@ setMethod("rand", signature(seed = "missing"), setMethod("rand", signature(seed =

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82908825 --- Diff: R/pkg/R/functions.R --- @@ -2876,7 +2897,8 @@ setMethod("randn", signature(seed = "missing"), setMethod("randn", signature(seed =

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82908731 --- Diff: R/pkg/R/functions.R --- @@ -2836,7 +2845,11 @@ setMethod("lpad", signature(x = "Column", len = "numeric", pad = "character"),

[GitHub] spark issue #11336: [SPARK-9325][SPARK-R] collect() head() and show() for Co...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11336 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66767/ Test FAILed. ---

[GitHub] spark issue #11336: [SPARK-9325][SPARK-R] collect() head() and show() for Co...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11336 **[Test build #66767 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66767/consoleFull)** for PR 11336 at commit

[GitHub] spark issue #11336: [SPARK-9325][SPARK-R] collect() head() and show() for Co...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11336 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82908439 --- Diff: R/pkg/R/column.R --- @@ -32,35 +34,65 @@ setOldClass("jobj") #' @export #' @note Column since 1.4.0 setClass("Column", -

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-11 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/14690 Btw I also made https://github.com/VideoAmp/spark-public/pull/2/files, to fix inputFiles. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82907977 --- Diff: R/pkg/R/DataFrame.R --- @@ -1168,12 +1179,14 @@ setMethod("take", #' Head #' -#' Return the first \code{num} rows of a

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82907897 --- Diff: R/pkg/R/DataFrame.R --- @@ -1182,10 +1195,18 @@ setMethod("take", #' @export #' @examples #'\dontrun{ -#' sparkR.session()

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82907729 --- Diff: R/pkg/R/DataFrame.R --- @@ -1035,10 +1035,16 @@ setMethod("dim", c(count(x), ncol(x)) }) -#' Collects

[GitHub] spark pull request #15389: [SPARK-17817][PySpark] PySpark RDD Repartitioning...

2016-10-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15389#discussion_r82907726 --- Diff: python/pyspark/rdd.py --- @@ -2029,7 +2028,15 @@ def coalesce(self, numPartitions, shuffle=False): >>> sc.parallelize([1, 2, 3, 4,

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-11 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15421 I suspect that it could be related to my R installation: localhost:~ mwang$ R R version 3.3.0 (2016-05-03) -- "Supposedly Educational" Copyright (C) 2016 The R Foundation

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-11 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15421 That's interesting. I patched your change to a clean checkout and simply tested against the example on the JIRA. It throws the above exception. val obj = (sqlSerDe._1)(dis,

[GitHub] spark pull request #15432: [SPARK-17854][SQL] rand/randn allows null as inpu...

2016-10-11 Thread HyukjinKwon
GitHub user HyukjinKwon reopened a pull request: https://github.com/apache/spark/pull/15432 [SPARK-17854][SQL] rand/randn allows null as input seed ## What changes were proposed in this pull request? This PR proposes `rand`/`randn` accept `null` as input. In this case, it

[GitHub] spark issue #15338: [SPARK-11653][Deploy] Allow spark-daemon.sh to run in th...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15338 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66759/ Test FAILed. ---

[GitHub] spark issue #15338: [SPARK-11653][Deploy] Allow spark-daemon.sh to run in th...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15338 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15432: [SPARK-17854][SQL] rand/randn allows null as input seed

2016-10-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15432 @rxin yes, I just wanted to avoid changing a lot. Will try to fix it in that way (at least) to show how it actually look like. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #15432: [SPARK-17854][SQL] rand/randn allows null as input seed

2016-10-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15432 @rxin yes, I just wanted to avoid changing a lot. Will try to fix it in that way (at least) to show how it actually look like. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #15432: [SPARK-17854][SQL] rand/randn allows null as inpu...

2016-10-11 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/15432 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #15338: [SPARK-11653][Deploy] Allow spark-daemon.sh to run in th...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15338 **[Test build #66759 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66759/consoleFull)** for PR 15338 at commit

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-11 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/15421 @wangmiao1981 I don't get the exception that you reported on Mac. Also note that the unit test is passing on Linux. I am not sure why returning null is an issue. --- If your project is set up for

[GitHub] spark issue #15375: [SPARK-17790][SPARKR] Support for parallelizing R data.f...

2016-10-11 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/15375 @felixcheung does it look OK now? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15408: [SPARK-17839][CORE] Use Nio's directbuffer instea...

2016-10-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15408#discussion_r82897143 --- Diff: core/src/main/java/org/apache/spark/io/NioBufferedFileInputStream.java --- @@ -0,0 +1,132 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #15408: [SPARK-17839][CORE] Use Nio's directbuffer instea...

2016-10-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15408#discussion_r82903865 --- Diff: core/src/main/java/org/apache/spark/io/NioBufferedFileInputStream.java --- @@ -0,0 +1,132 @@ +/* + * Licensed under the Apache License,

[GitHub] spark issue #11336: [SPARK-9325][SPARK-R] collect() head() and show() for Co...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11336 **[Test build #66767 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66767/consoleFull)** for PR 11336 at commit

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-11 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82902172 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -225,13 +225,16 @@ case class FileSourceScanExec( }

[GitHub] spark issue #15408: [SPARK-17839][CORE] Use Nio's directbuffer instead of Bu...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15408 **[Test build #66765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66765/consoleFull)** for PR 15408 at commit

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13440 **[Test build #66766 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66766/consoleFull)** for PR 13440 at commit

[GitHub] spark issue #15408: [SPARK-17839][CORE] Use Nio's directbuffer instead of Bu...

2016-10-11 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/15408 jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2016-10-11 Thread erikerlandson
Github user erikerlandson commented on the issue: https://github.com/apache/spark/pull/13440 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-11 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82900810 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -225,13 +225,16 @@ case class FileSourceScanExec( }

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14690 **[Test build #66764 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66764/consoleFull)** for PR 14690 at commit

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-11 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15421 I did test on Mac, not on Windows. I don't have windows setup either. The very first issue that I saw is the negative index issue. In addition, returning `null` when exception is

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15307 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66763/ Test FAILed. ---

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15307 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66763 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66763/consoleFull)** for PR 15307 at commit

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #3325 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3325/consoleFull)** for PR 15307 at commit

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66763 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66763/consoleFull)** for PR 15307 at commit

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #3325 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3325/consoleFull)** for PR 15307 at commit

[GitHub] spark pull request #14959: [SPARK-17387][PYSPARK] Creating SparkContext() fr...

2016-10-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14959 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-11 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/15421 My guess is that on windows R serialization behaves differently and serializes `NA` as `null`. Unfortunately, I don't have a windows machine to verify. Would you please test that? --- If your

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-10-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14959 LGTM, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #11336: [SPARK-9325][SPARK-R] collect() head() and show() for Co...

2016-10-11 Thread olarayej
Github user olarayej commented on the issue: https://github.com/apache/spark/pull/11336 @falaki Yeah but those warnings are making the build fail (see below). Is that okay? Now I see a new"Checks" section. I may be outdated with the protocols as it's been a while I didn't commit :-).

[GitHub] spark issue #11336: [SPARK-9325][SPARK-R] collect() head() and show() for Co...

2016-10-11 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/11336 I think some of the old tests still rely on `sparkRSQL.init()`. I believe the warning is OK. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #15422: [SPARK-17850][Core]HadoopRDD should not catch EOFExcepti...

2016-10-11 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15422 @srowen Since this is happening 'below' the user code (in the hadoop rdd), is there a way around how to handle this ? I agree that for a lot of usecases where it is critical to work off the

[GitHub] spark issue #15422: [SPARK-17850][Core]HadoopRDD should not catch EOFExcepti...

2016-10-11 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15422 @zsxwing The map task is run by

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15421 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15421 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66755/ Test PASSed. ---

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15421 **[Test build #66755 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66755/consoleFull)** for PR 15421 at commit

[GitHub] spark issue #15389: [SPARK-17817][PySpark] PySpark RDD Repartitioning Result...

2016-10-11 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15389 @felixcheung merged it I believe. @felixcheung please make sure you leave a msg saying it's merged (along with the branch) when you merge prs. --- If your project is set up for it, you can

[GitHub] spark pull request #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSet...

2016-10-11 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/15249#discussion_r82891721 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -98,6 +84,14 @@ private[spark] class TaskSetManager( var

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82890911 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamMetrics.scala --- @@ -0,0 +1,244 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #15389: [SPARK-17817][PySpark] PySpark RDD Repartitioning Result...

2016-10-11 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15389 Did not release that we already merged this, should we left a message here or in the JIRA so we can know who merge this? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82890581 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -516,12 +563,127 @@ class StreamExecution(

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82890465 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -105,11 +105,21 @@ class StreamExecution( var

[GitHub] spark issue #11336: [SPARK-9325][SPARK-R] collect() head() and show() for Co...

2016-10-11 Thread olarayej
Github user olarayej commented on the issue: https://github.com/apache/spark/pull/11336 @falaki Thanks for your comments. Yeah, before removing collect/show, I just wanted to rebase to current upstream master. I'm getting a build error which is actually a warning, not even an R

[GitHub] spark pull request #15389: [SPARK-17817][PySpark] PySpark RDD Repartitioning...

2016-10-11 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/15389#discussion_r82890280 --- Diff: python/pyspark/rdd.py --- @@ -2029,7 +2028,15 @@ def coalesce(self, numPartitions, shuffle=False): >>> sc.parallelize([1, 2, 3, 4, 5],

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15438 **[Test build #66762 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66762/consoleFull)** for PR 15438 at commit

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82890127 --- Diff: python/pyspark/sql/streaming.py --- @@ -189,6 +189,282 @@ def resetTerminated(self): self._jsqm.resetTerminated()

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82890125 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -176,7 +184,9 @@ class StreamExecution( //

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread falaki
Github user falaki commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82888215 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -2252,6 +2252,31 @@ test_that("Method str()", {

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread falaki
Github user falaki commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82887849 --- Diff: R/pkg/R/column.R --- @@ -32,35 +34,65 @@ setOldClass("jobj") #' @export #' @note Column since 1.4.0 setClass("Column", -

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread falaki
Github user falaki commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82888061 --- Diff: R/pkg/R/column.R --- @@ -32,35 +34,65 @@ setOldClass("jobj") #' @export #' @note Column since 1.4.0 setClass("Column", -

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-11 Thread falaki
Github user falaki commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r82887445 --- Diff: R/pkg/R/DataFrame.R --- @@ -1035,10 +1035,17 @@ setMethod("dim", c(count(x), ncol(x)) }) -#' Collects all

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13440 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13440 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66756/ Test FAILed. ---

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13440 **[Test build #66756 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66756/consoleFull)** for PR 13440 at commit

[GitHub] spark issue #15437: [SPARK-17876] Write StructuredStreaming WAL to a stream ...

2016-10-11 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15437 cc @tdas @zsxwing Would one of you want to look at this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14959 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14959 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66758/ Test PASSed. ---

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14959 **[Test build #66758 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66758/consoleFull)** for PR 14959 at commit

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15438 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66761/ Test FAILed. ---

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15438 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15438 **[Test build #66761 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66761/consoleFull)** for PR 15438 at commit

[GitHub] spark issue #12576: [SPARK-14804][Spark][Graphx] Fix Graph vertexRDD/EdgeRDD...

2016-10-11 Thread neggert
Github user neggert commented on the issue: https://github.com/apache/spark/pull/12576 Not sure what's going on regarding the multiple PRs for this issue, but I cherry-picked this PR on top of 1.6.2 and it fixed the problem for me. --- If your project is set up for it, you can reply

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15438 **[Test build #66760 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66760/consoleFull)** for PR 15438 at commit

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15438 **[Test build #66761 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66761/consoleFull)** for PR 15438 at commit

[GitHub] spark issue #15437: [SPARK-17876] Write StructuredStreaming WAL to a stream ...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15437 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66754/ Test PASSed. ---

[GitHub] spark issue #15437: [SPARK-17876] Write StructuredStreaming WAL to a stream ...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15437 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15437: [SPARK-17876] Write StructuredStreaming WAL to a stream ...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15437 **[Test build #66754 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66754/consoleFull)** for PR 15437 at commit

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15438 **[Test build #66760 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66760/consoleFull)** for PR 15438 at commit

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15438 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66760/ Test FAILed. ---

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15438 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15438: [SPARK-17845][SQL] More self-evident window function fra...

2016-10-11 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15438 cc @thunterdb, @hvanhovell, @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15438: [SPARK-17845][SQL] More self-evident window funct...

2016-10-11 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/15438 [SPARK-17845][SQL] More self-evident window function frame boundary API ## What changes were proposed in this pull request? This patch improves the window function frame boundary API to make

[GitHub] spark issue #15436: [SPARK-17875] [BUILD] Remove unneeded direct dependence ...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15436 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66752/ Test FAILed. ---

[GitHub] spark issue #15436: [SPARK-17875] [BUILD] Remove unneeded direct dependence ...

2016-10-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15436 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15436: [SPARK-17875] [BUILD] Remove unneeded direct dependence ...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15436 **[Test build #66752 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66752/consoleFull)** for PR 15436 at commit

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15190 Added them to your test cases. : ) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15338: [SPARK-11653][Deploy] Allow spark-daemon.sh to run in th...

2016-10-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15338 **[Test build #66759 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66759/consoleFull)** for PR 15338 at commit

[GitHub] spark issue #15338: [SPARK-11653][Deploy] Allow spark-daemon.sh to run in th...

2016-10-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15338 Jenkins add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15425: [SPARK-17816] [Core] [Branch-2.0] Fix ConcurrentM...

2016-10-11 Thread seyfe
Github user seyfe closed the pull request at: https://github.com/apache/spark/pull/15425 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15425: [SPARK-17816] [Core] [Branch-2.0] Fix ConcurrentModifica...

2016-10-11 Thread seyfe
Github user seyfe commented on the issue: https://github.com/apache/spark/pull/15425 Closing it since it's merged into 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15382: [SPARK-17810] [SQL] Default spark.sql.warehouse.d...

2016-10-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15382#discussion_r82878712 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -757,7 +758,10 @@ private[sql] class SQLConf extends Serializable with

<    1   2   3   4   5   6   7   >