[GitHub] spark pull request #14976: [SPARK-17306] [SQL] QuantileSummaries doesn't com...

2016-09-06 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14976#discussion_r77726882 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/QuantileSummaries.scala --- @@ -59,9 +59,14 @@ class QuantileSummaries( *

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-06 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r77724521 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-06 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r77723977 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/14961 > Is the lesson here to not bother with pooling and use the UnpooledByteBufAllocator? Not sure. Pooling is for improving the performance because allocating direct buffers is pretty slow.

[GitHub] spark pull request #14943: [SPARK-15891][yarn] Clean up some logging in the ...

2016-09-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14943#discussion_r77722666 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala --- @@ -59,43 +58,46 @@ private[yarn] class ExecutorRunnable( v

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14961 Aha, possibly this: https://groups.google.com/forum/#!topic/netty/3BoF7q34Z4I Is the lesson here to not bother with pooling and use the UnpooledByteBufAllocator? --- If your project is

[GitHub] spark issue #14943: [SPARK-15891][yarn] Clean up some logging in the YARN AM...

2016-09-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14943 +1 feel free to commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark issue #14943: [SPARK-15891][yarn] Clean up some logging in the YARN AM...

2016-09-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14943 +1 feel free to commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark issue #14943: [SPARK-15891][yarn] Clean up some logging in the YARN AM...

2016-09-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14943 +1 feel free to commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark pull request #14943: [SPARK-15891][yarn] Clean up some logging in the ...

2016-09-06 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14943#discussion_r77722010 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala --- @@ -59,43 +58,46 @@ private[yarn] class ExecutorRunnable(

[GitHub] spark issue #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression interf...

2016-09-06 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14834 @jkbradley Thanks for your input. Let's see what @dbtsai thinks as well :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark issue #14927: [SPARK-16922] [SPARK-17211] [SQL] make the address of va...

2016-09-06 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14927 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark pull request #14972: [Minor] Remove unnecessary check in MLSerDe

2016-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14972 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #14924: [SPARK-17299] TRIM/LTRIM/RTRIM should not strips ...

2016-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14924 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14972: [Minor] Remove unnecessary check in MLSerDe

2016-09-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/14972 I'm OK with this not having a JIRA, and I agree the code path is tested sufficiently. (But wanting these is a good sentiment!) LGTM Merging with master --- If your project is set up for

[GitHub] spark issue #14924: [SPARK-17299] TRIM/LTRIM/RTRIM should not strips charact...

2016-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14924 Merged to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14924: [SPARK-17299] TRIM/LTRIM/RTRIM should not strips charact...

2016-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14924 Merged to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-09-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14750 We still need to resolve the case sensitivity issues in [`checkDuplication`](https://github.com/apache/spark/blob/6d86403d8b252776effcddd71338b4d21a224f9b/sql/core/src/main/scala/org/apache/spark/

[GitHub] spark issue #14958: [SPARK-17378] [BUILD] Upgrade snappy-java to 1.1.2.6

2016-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14958 Merged to master/2.0/1.6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #14958: [SPARK-17378] [BUILD] Upgrade snappy-java to 1.1....

2016-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14958 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r77720376 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #14983: [SPARK-17316][Core]Fix the 'ask' type parameter in 'remo...

2016-09-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14983 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-06 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r77719621 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #14983: [SPARK-17316][Core]Fix the 'ask' type parameter in 'remo...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14983 **[Test build #65010 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65010/consoleFull)** for PR 14983 at commit [`1ba43d6`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #13036: [SPARK-15243][ML][SQL][PYSPARK] Param methods should use...

2016-09-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/13036 Checking old PRs---is this active still? The ML parts look good to me. I haven't checked the SQL ones carefully. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #14983: [SPARK-17316][Core]Fix the 'ask' type parameter in 'remo...

2016-09-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/14983 cc @vanzin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #14983: [SPARK-17316][Core]Fix the 'ask' type parameter i...

2016-09-06 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/14983 [SPARK-17316][Core]Fix the 'ask' type parameter in 'removeExecutor' ## What changes were proposed in this pull request? Fix the 'ask' type parameter in 'removeExecutor' to eliminate a lot o

[GitHub] spark issue #14981: [SPARK-17418] Remove Kinesis artifacts from Spark releas...

2016-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14981 Leave the ganglia one because that artifact isn't an assembly after all. Publishing the Spark part of that without redistributing the rest is OK. Really, here we need to only pull the kinesi

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/14961 I saw the error in the log: ``` 16/09/05 08:21:56.758 shuffle-server-593-8 WARN TransportChannelHandler: Exception in connection from /127.0.0.1:44788 io.netty.util.internal.OutOfDirectMe

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13762 **[Test build #3251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3251/consoleFull)** for PR 13762 at commit [`22a3a04`](https://github.com/apache/spark/commit

[GitHub] spark issue #14112: [SPARK-16240][ML] Model loading backward compatibility f...

2016-09-06 Thread GayathriMurali
Github user GayathriMurali commented on the issue: https://github.com/apache/spark/pull/14112 @jkbradley I am so sorry I couldn't respond to this on time! I am in a transition process and might not be able to drive this JIRA to completion at this point in time. Thanks! --- If your

[GitHub] spark issue #14981: [SPARK-17418] Remove Kinesis artifacts from Spark releas...

2016-09-06 Thread lresende
Github user lresende commented on the issue: https://github.com/apache/spark/pull/14981 As for the the Ganglia one, I will create another jira, to track that separately as this (Kinesis) one might involve more changes around the python and examples. --- If your project is set up for

[GitHub] spark issue #13493: [SPARK-15750][MLLib][PYSPARK] Constructing FPGrowth fail...

2016-09-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/13493 Pinging! Is this still active? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13762 **[Test build #3251 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3251/consoleFull)** for PR 13762 at commit [`22a3a04`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14981: [SPARK-17418] Remove Kinesis artifacts from Spark releas...

2016-09-06 Thread lresende
Github user lresende commented on the issue: https://github.com/apache/spark/pull/14981 @srowen, The Kinesis assembly has been published by Spark releases for a while. Here is the link to the 2.0 release on repository.apache.org : https://repository.apache.org/service/local/reposi

[GitHub] spark issue #14112: [SPARK-16240][ML] Model loading backward compatibility f...

2016-09-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/14112 @GayathriMurali Btw, I know it's been a long time. If you no longer have time to work on this, please say so, and I'd be happy to take it over. Thanks! --- If your project is set up for it, you

[GitHub] spark issue #9440: [Spark-11478] [ML] ML StringIndexer return inconsistent s...

2016-09-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/9440 Is this still active? If not, can you please close this issue pending discussion on the JIRA? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression interf...

2016-09-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/14834 +1 for Option 2: Store the binomial coefficients as a 1 x numFeatures matrix. It's such an important code path that I think it's worth avoiding the regression for current users. --- If

[GitHub] spark issue #14979: [SPARK-17415][SQL] Better error message for driver-side ...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14979 **[Test build #3250 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3250/consoleFull)** for PR 14979 at commit [`b3c5cfe`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark.sql.hi...

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14978 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65004/ Test PASSed. ---

[GitHub] spark issue #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark.sql.hi...

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14978 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark.sql.hi...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14978 **[Test build #65004 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65004/consoleFull)** for PR 14978 at commit [`fab1730`](https://github.com/apache/spark/commit/

[GitHub] spark issue #10970: [SPARK-13067][SQL] workaround for a weird scala reflecti...

2016-09-06 Thread atronchi
Github user atronchi commented on the issue: https://github.com/apache/spark/pull/10970 This solution does not seem to work for datasets on tables using complex types. e.g. Array[Int] still throws the exception `java.lang.AssertionError: assertion failed: Unsound substitution from Lis

[GitHub] spark issue #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark.sql.hi...

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14978 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark.sql.hi...

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14978 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65003/ Test PASSed. ---

[GitHub] spark issue #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark.sql.hi...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14978 **[Test build #65003 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65003/consoleFull)** for PR 14978 at commit [`a257bc5`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14981: [SPARK-17418] Remove Kinesis artifacts from Spark releas...

2016-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14981 Aha, right. That one is probably OK as there's technically no third-party code distributed in those artifacts. However, this won't be OK: https://mvnrepository.com/artifact/org.apache.spark/spark-str

[GitHub] spark issue #14979: [SPARK-17415][SQL] Better error message for driver-side ...

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14979 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65005/ Test FAILed. ---

[GitHub] spark issue #14979: [SPARK-17415][SQL] Better error message for driver-side ...

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14979 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14979: [SPARK-17415][SQL] Better error message for driver-side ...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14979 **[Test build #65005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65005/consoleFull)** for PR 14979 at commit [`b3c5cfe`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14981: [SPARK-17418] Remove Kinesis artifacts from Spark releas...

2016-09-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14981 They're not in the assembly, but they're published to maven as a separate thing; can that be an issue? https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kinesis-asl_2.10 --- I

[GitHub] spark issue #14981: [SPARK-17418] Remove Kinesis artifacts from Spark releas...

2016-09-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14981 @lresende I don't think we actually distribute the kinesis code, because that profile just enables the kinesis assembly, and that does not cause this stuff to get built into a Spark jar that's distri

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14980 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14980 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65007/ Test FAILed. ---

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14980 **[Test build #65007 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65007/consoleFull)** for PR 14980 at commit [`4386ab3`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14982: [WIP] Not re-use dictionary across batches

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14982 **[Test build #65009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65009/consoleFull)** for PR 14982 at commit [`eab4b63`](https://github.com/apache/spark/commit/e

[GitHub] spark pull request #14982: [WIP] Not re-use dictionary across batches

2016-09-06 Thread sameeragarwal
GitHub user sameeragarwal opened a pull request: https://github.com/apache/spark/pull/14982 [WIP] Not re-use dictionary across batches You can merge this pull request into a Git repository by running: $ git pull https://github.com/sameeragarwal/spark parquet-repro Alternative

[GitHub] spark issue #14981: [SPARK-17418] Remove Kinesis artifacts from Spark releas...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14981 **[Test build #65008 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65008/consoleFull)** for PR 14981 at commit [`375d62e`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #14981: [SPARK-17418] Remove Kinesis artifacts from Spark...

2016-09-06 Thread lresende
GitHub user lresende opened a pull request: https://github.com/apache/spark/pull/14981 [SPARK-17418] Remove Kinesis artifacts from Spark release scripts ## What changes were proposed in this pull request? This PR removes Kinesis from the release scripts as Kinesis license (Amazon

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14980 **[Test build #65007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65007/consoleFull)** for PR 14980 at commit [`4386ab3`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14980 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65006/ Test PASSed. ---

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14980 **[Test build #65006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65006/consoleFull)** for PR 14980 at commit [`cbd0b41`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14980 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-06 Thread junyangq
Github user junyangq commented on the issue: https://github.com/apache/spark/pull/14980 cc @shivaram @felixcheung @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14980 **[Test build #65006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65006/consoleFull)** for PR 14980 at commit [`cbd0b41`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #14952: [SPARK-17110] Fix StreamCorruptionException in Bl...

2016-09-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/14952#discussion_r77699493 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -520,10 +520,11 @@ private[spark] class BlockManager( * *

[GitHub] spark pull request #14980: [SPARK-17317][SparkR] Add SparkR vignette

2016-09-06 Thread junyangq
GitHub user junyangq opened a pull request: https://github.com/apache/spark/pull/14980 [SPARK-17317][SparkR] Add SparkR vignette ## What changes were proposed in this pull request? This PR tries to add a SparkR vignette, which works as a friendly guidance going through the

[GitHub] spark issue #14742: [SPARK-17177][SQL] Make grouping columns accessible from...

2016-09-06 Thread NarineK
Github user NarineK commented on the issue: https://github.com/apache/spark/pull/14742 @liancheng, @rxin, Do you think adding `columns` to `RelationalGroupedDataset` is reasonable or should we find a workaround on R side ? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request #14952: [SPARK-17110] Fix StreamCorruptionException in Bl...

2016-09-06 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/14952#discussion_r77697794 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -520,10 +520,11 @@ private[spark] class BlockManager( *

[GitHub] spark pull request #14959: [SPARK-17387][PYSPARK] Creating SparkContext() fr...

2016-09-06 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/14959#discussion_r77697522 --- Diff: python/pyspark/conf.py --- @@ -101,13 +101,31 @@ def __init__(self, loadDefaults=True, _jvm=None, _jconf=None): self._jconf =

[GitHub] spark issue #14979: [SPARK-17415][SQL] Better error message for driver-side ...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14979 **[Test build #65005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65005/consoleFull)** for PR 14979 at commit [`b3c5cfe`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14979: [SPARK-17415][SQL] Better error message for driver-side ...

2016-09-06 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/14979 cc @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark.sql.hi...

2016-09-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14978 cc @yhuai @cloud-fan @hvanhovell @wzhfy --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14979: Better error message for driver-side broadcast join OOMs

2016-09-06 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/14979 Here is the list of changes with whitespaces ignored: https://github.com/apache/spark/pull/14979/files?w=1 --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] spark issue #14952: [SPARK-17110] Fix StreamCorruptionException in BlockMana...

2016-09-06 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/14952 looks good --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] spark pull request #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark...

2016-09-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14978#discussion_r77696450 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -171,23 +171,37 @@ class StatisticsSuite extends QueryTest with

[GitHub] spark issue #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark.sql.hi...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14978 **[Test build #65004 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65004/consoleFull)** for PR 14978 at commit [`fab1730`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #14979: Better error message for driver-side broadcast jo...

2016-09-06 Thread sameeragarwal
GitHub user sameeragarwal opened a pull request: https://github.com/apache/spark/pull/14979 Better error message for driver-side broadcast join OOMs ## What changes were proposed in this pull request? This is a trivial patch that catches all `OutOfMemoryError` while building

[GitHub] spark issue #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark.sql.hi...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14978 **[Test build #65003 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65003/consoleFull)** for PR 14978 at commit [`a257bc5`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #14978: [SPARK-17408] [TEST] Flaky test: org.apache.spark...

2016-09-06 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/14978 [SPARK-17408] [TEST] Flaky test: org.apache.spark.sql.hive.StatisticsSuite ### What changes were proposed in this pull request? https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBui

[GitHub] spark pull request #14960: [SPARK-17339][SPARKR][CORE] Fix some R tests and ...

2016-09-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14960#discussion_r77694632 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -992,7 +992,7 @@ class SparkContext(config: SparkConf) extends Logging with Exec

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14809 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65002/ Test PASSed. ---

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14809 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14809 **[Test build #65002 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65002/consoleFull)** for PR 14809 at commit [`7caf2fb`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14958: [SPARK-17378] [BUILD] Upgrade snappy-java to 1.1.2.6

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14958 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14958: [SPARK-17378] [BUILD] Upgrade snappy-java to 1.1.2.6

2016-09-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14958 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65001/ Test PASSed. ---

[GitHub] spark issue #14958: [SPARK-17378] [BUILD] Upgrade snappy-java to 1.1.2.6

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14958 **[Test build #65001 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65001/consoleFull)** for PR 14958 at commit [`022cad7`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14927: [SPARK-16922] [SPARK-17211] [SQL] make the addres...

2016-09-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14927 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14944: [SPARK-16334][BACKPORT] Reusing same dictionary column f...

2016-09-06 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14944 Merging this into 2.0 branch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r77684425 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-06 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r77684087 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #14927: [SPARK-16922] [SPARK-17211] [SQL] make the address of va...

2016-09-06 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14927 @hvanhovell Has lgtm offline. Other contributors also confirm that this patch fix the bug, I'm going to merge this one into master and 2.0 branch. --- If your project is set up for it, you can reply

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-06 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r77683485 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-09-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14712 These tests block multiple PRs. It is midnight in China. : ) Let me do a quick fix based on the comments of @cloud-fan and @hvanhovell --- If your project is set up for it, you can reply to thi

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-06 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r77680444 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #14973: [SPARK-17356][SQL][1.6] Fix out of memory issue when gen...

2016-09-06 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14973 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feat

[GitHub] spark issue #14968: [SPARK-17369][SQL][2.0] MetastoreRelation toJSON throws ...

2016-09-06 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14968 @clockfly Can you close this? It has been merged to branch 2.0 (btw, prs targeting branches other than master will not be auto-closed). --- If your project is set up for it, you can reply to this em

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-09-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14809 **[Test build #65002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65002/consoleFull)** for PR 14809 at commit [`7caf2fb`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-09-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14809 LGTM pending test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-09-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14809 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

<    1   2   3   4   5   6   >