[GitHub] spark pull request: [SQL] [Minor] Loose the the resolving check
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5224#issuecomment-86847202 [Test build #29290 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29290/consoleFull) for PR 5224 at commit [`7dbc303`](https://github.com/apache/spark/commit/7dbc303870f04a3d99cd7a807cf4dffeab7db8b5). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL] [Minor] Loose the the resolving check
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/5224 [SQL] [Minor] Loose the the resolving check You can merge this pull request into a Git repository by running: $ git pull https://github.com/chenghao-intel/spark patch-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/5224.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5224 commit 7dbc303870f04a3d99cd7a807cf4dffeab7db8b5 Author: Cheng Hao Date: 2015-03-27T06:51:11Z Loose the the resolving check --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6556][Core] Fix wrong parsing logic of ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5209#issuecomment-86846222 [Test build #29288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29288/consoleFull) for PR 5209 at commit [`6a0a411`](https://github.com/apache/spark/commit/6a0a411578f8e72b42311bf32bda62c75b2956ef). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6556][Core] Fix wrong parsing logic of ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5209#issuecomment-86846252 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29288/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/5063#issuecomment-86841550 @sryza Review it again, please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5063#issuecomment-86841449 [Test build #29285 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29285/consoleFull) for PR 5063 at commit [`8c3c065`](https://github.com/apache/spark/commit/8c3c0652a5cbb78921324bb3bd0a4ea4216c167f). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5063#issuecomment-86841450 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29285/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4897] [PySpark] Python 3 support
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5173#issuecomment-86840184 [Test build #12 has started](https://hadrian.millennium.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/12/consoleFull) for PR 5173 at commit [`59bb492`](https://github.com/apache/spark/commit/59bb49260f62fda2af0d48e35447d1a7dcd0a479). * This patch **does not merge cleanly**. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4897] [PySpark] Python 3 support
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5173#issuecomment-86838162 [Test build #29284 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29284/consoleFull) for PR 5173 at commit [`59bb492`](https://github.com/apache/spark/commit/59bb49260f62fda2af0d48e35447d1a7dcd0a479). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4897] [PySpark] Python 3 support
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5173#issuecomment-86838175 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29284/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6526][ML] Add Normalizer transformer in...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5181#issuecomment-86835479 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29283/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6526][ML] Add Normalizer transformer in...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5181#issuecomment-86835463 [Test build #29283 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29283/consoleFull) for PR 5181 at commit [`6faa7bf`](https://github.com/apache/spark/commit/6faa7bfdfd06d0b86fa1ad9e8d2c3722d2e90435). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Normalizer extends UnaryTransformer[Vector, Vector, Normalizer] ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [spark] [SPARK-6168] Expose some of the collec...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5084#issuecomment-86835246 Yeah I agree with Sean and what pretty much everyone else said. My feeling is that we don't want typical spark users to be relying on unstable API's, else it creates a lot of upgrade friction for users and/or pressure on maintainers to stabilize this stuff. We annotate things as unstable and expose them for two reasons (a) to get feedback on new API's we intend to stabilize shortly (b) to expose internal hooks that are meant for third party integrations (similar to kernel modules) such as data sources, but not expected to be needed by end users. Exposing internal utilities for end users, IMO, it's just not a good idea. I don't think slapping an annotation means that there is no cost to doing this. Users will rely on this stuff and it will create upgrade friction, and this is why we are careful about what we expose. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5218 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6341][mllib] Upgrade breeze from 0.11.1...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5222#issuecomment-86834829 [Test build #29281 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29281/consoleFull) for PR 5222 at commit [`ad8a688`](https://github.com/apache/spark/commit/ad8a688e980d7c2cbd5d16cbc146f7fb85934cbe). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5218#issuecomment-86834844 LGTM - merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6341][mllib] Upgrade breeze from 0.11.1...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5222#issuecomment-86834834 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29281/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6560][CORE] Do not suppress exceptions ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5223#issuecomment-86834742 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29280/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6560][CORE] Do not suppress exceptions ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5223#issuecomment-86834738 [Test build #29280 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29280/consoleFull) for PR 5223 at commit [`f42e92d`](https://github.com/apache/spark/commit/f42e92de46967e0b8dc19c198ea0ea6196c6ebca). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class RegexTokenizer extends UnaryTransformer[String, Seq[String], RegexTokenizer] ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-86834640 [Test build #29289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29289/consoleFull) for PR 3247 at commit [`40fed21`](https://github.com/apache/spark/commit/40fed21254ae0625b708727b4cfc858686548924). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class UnresolvedFunction(` * `trait AggregateFunction ` * `trait AggregateExpression extends Expression with AggregateFunction ` * `abstract class UnaryAggregateExpression extends UnaryExpression with AggregateExpression ` * `case class Min(` * `case class Average(child: Expression, distinct: Boolean = false)` * `case class Max(child: Expression)` * `case class Count(child: Expression)` * `case class CountDistinct(children: Seq[Expression])` * `case class Sum(child: Expression, distinct: Boolean = false)` * `case class First(child: Expression, distinct: Boolean = false)` * `case class Last(child: Expression, distinct: Boolean = false)` * `sealed case class AggregateFunctionBind(` * `sealed class InputBufferSeens(` * `sealed trait Aggregate ` * `sealed trait PreShuffle extends Aggregate ` * `sealed trait PostShuffle extends Aggregate ` * `case class AggregatePreShuffle(` * `case class AggregatePostShuffle(` * `case class DistinctAggregate(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-86834645 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29289/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][SPARK-6471]: Metastore schema should onl...
Github user saucam commented on the pull request: https://github.com/apache/spark/pull/5141#issuecomment-86832254 Hi @liancheng , We do have use cases where 100K partitions will be registered in tables, (partitioned on timestamps, data is added in form of partitions for every 5min interval) , but it could be more in other cases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5124#discussion_r27276853
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -474,13 +477,13 @@ class SparkContext(config: SparkConf) extends Logging
with ExecutorAllocationCli
* Spark fair scheduler pool.
*/
def setLocalProperty(key: String, value: String) {
-if (localProperties.get() == null) {
- localProperties.set(new Properties())
+if (localProperties.get().isEmpty) {
+ localProperties.set(Some(new Properties()))
--- End diff --
Basically, this behavior of having `localProperties` be null or None until
we set at least one property is confusing to me; I think it's simpler to just
eagerly perform this initialization before we set any properties.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5124#discussion_r27276816
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -474,13 +477,13 @@ class SparkContext(config: SparkConf) extends Logging
with ExecutorAllocationCli
* Spark fair scheduler pool.
*/
def setLocalProperty(key: String, value: String) {
-if (localProperties.get() == null) {
- localProperties.set(new Properties())
+if (localProperties.get().isEmpty) {
+ localProperties.set(Some(new Properties()))
--- End diff --
Instead of having `localProperties` be an `Option[Property]`, why not just
have `override def initialValue(): Properties = new Properties()`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6113] [mllib] [WIP] Stabilize DecisionT...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5009#issuecomment-86830278 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29279/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6556][Core] Fix wrong parsing logic of ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5209#issuecomment-86830111 [Test build #29288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29288/consoleFull) for PR 5209 at commit [`6a0a411`](https://github.com/apache/spark/commit/6a0a411578f8e72b42311bf32bda62c75b2956ef). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6113] [mllib] [WIP] Stabilize DecisionT...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5009#issuecomment-86830245 [Test build #29279 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29279/consoleFull) for PR 5009 at commit [`7534133`](https://github.com/apache/spark/commit/75341330e79e6f8f96f33469a92f700f187e5eaa). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-86830088 [Test build #29289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29289/consoleFull) for PR 3247 at commit [`40fed21`](https://github.com/apache/spark/commit/40fed21254ae0625b708727b4cfc858686548924). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6556][Core] Fix wrong parsing logic of ...
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/5209#issuecomment-86829270 Fixed the docs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6521][Core]executors in the same node r...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5178#issuecomment-86828278 **[Test build #29274 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29274/consoleFull)** for PR 5178 at commit [`bb94736`](https://github.com/apache/spark/commit/bb9473641c575e5022b95ee89372fe8981aa0814) after a configured wait of `120m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6521][Core]executors in the same node r...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5178#issuecomment-86828291 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29274/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-86827534 [Test build #29287 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29287/consoleFull) for PR 3247 at commit [`70117c4`](https://github.com/apache/spark/commit/70117c4e4802c1cc2050a7ed5ae2f72ca13d5c02). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class UnresolvedFunction(` * `trait AggregateFunction ` * `trait AggregateExpression extends Expression with AggregateFunction ` * `abstract class UnaryAggregateExpression extends UnaryExpression with AggregateExpression ` * `case class Min(` * `case class Average(child: Expression, distinct: Boolean = false)` * `case class Max(child: Expression)` * `case class Count(child: Expression)` * `case class CountDistinct(children: Seq[Expression])` * `case class Sum(child: Expression, distinct: Boolean = false)` * `case class First(child: Expression, distinct: Boolean = false)` * `case class Last(child: Expression, distinct: Boolean = false)` * `//case class Add(left: Expression, right: Expression) extends BinaryArithmetic ` * `sealed case class AggregateFunctionBind(` * `sealed class InputBufferSeens(` * `sealed trait Aggregate ` * `sealed trait PreShuffle extends Aggregate ` * `sealed trait PostShuffle extends Aggregate ` * `case class AggregatePreShuffle(` * `case class AggregatePostShuffle(` * `case class DistinctAggregate(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-86827541 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29287/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-86827254 [Test build #29287 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29287/consoleFull) for PR 3247 at commit [`70117c4`](https://github.com/apache/spark/commit/70117c4e4802c1cc2050a7ed5ae2f72ca13d5c02). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2475][Streaming] Check whether has enou...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5212#issuecomment-86827320 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29278/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2475][Streaming] Check whether has enou...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5212#issuecomment-86827291 [Test build #29278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29278/consoleFull) for PR 5212 at commit [`197a030`](https://github.com/apache/spark/commit/197a0304c70ef5262e30065fbec12595c1c72cb1). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-86826895 [Test build #29286 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29286/consoleFull) for PR 3247 at commit [`0915435`](https://github.com/apache/spark/commit/09154354bcf3c552497e8a6aac9bb84ee38a6aa3). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class UnresolvedFunction(` * `trait AggregateFunction ` * `trait AggregateExpression extends Expression with AggregateFunction ` * `abstract class UnaryAggregateExpression extends UnaryExpression with AggregateExpression ` * `case class Min(` * `case class Average(child: Expression, distinct: Boolean = false)` * `case class Max(child: Expression)` * `case class Count(child: Expression)` * `case class CountDistinct(children: Seq[Expression])` * `case class Sum(child: Expression, distinct: Boolean = false)` * `case class First(child: Expression, distinct: Boolean = false)` * `case class Last(child: Expression, distinct: Boolean = false)` * `//case class Add(left: Expression, right: Expression) extends BinaryArithmetic ` * `sealed case class AggregateFunctionBind(` * `sealed class InputBufferSeens(` * `sealed trait Aggregate ` * `sealed trait PreShuffle extends Aggregate ` * `sealed trait PostShuffle extends Aggregate ` * `case class AggregatePreShuffle(` * `case class AggregatePostShuffle(` * `case class DistinctAggregate(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-86826897 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29286/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-86826822 [Test build #29286 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29286/consoleFull) for PR 3247 at commit [`0915435`](https://github.com/apache/spark/commit/09154354bcf3c552497e8a6aac9bb84ee38a6aa3). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-86826358 Sorry, @maropu I'v updated. Let's see if will break anything. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5063#issuecomment-86824172 [Test build #29285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29285/consoleFull) for PR 5063 at commit [`8c3c065`](https://github.com/apache/spark/commit/8c3c0652a5cbb78921324bb3bd0a4ea4216c167f). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5218#issuecomment-86823880 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29277/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5218#issuecomment-86823849 [Test build #29277 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29277/consoleFull) for PR 5218 at commit [`1d6d1be`](https://github.com/apache/spark/commit/1d6d1be9ef335c51b477d8c2f661a72cd1c82f73). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6559][PySpark]specify port of python ga...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5221#issuecomment-86822200 [Test build #29276 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29276/consoleFull) for PR 5221 at commit [`1e5c85d`](https://github.com/apache/spark/commit/1e5c85d7480c6ca91381c8c1629653d4943fc638). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6559][PySpark]specify port of python ga...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5221#issuecomment-8687 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29276/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4897] [PySpark] Python 3 support
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5173#issuecomment-86822202 [Test build #29284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29284/consoleFull) for PR 5173 at commit [`59bb492`](https://github.com/apache/spark/commit/59bb49260f62fda2af0d48e35447d1a7dcd0a479). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Github user jongyoul commented on a diff in the pull request: https://github.com/apache/spark/pull/5063#discussion_r27275980 --- Diff: docs/running-on-mesos.md --- @@ -211,6 +211,14 @@ See the [configuration page](configuration.html) for information on Spark config + spark.mesos.executor.cores + 1.0 + +The setting, which can be a floating point number, controls the number of cores allocated --- End diff -- Ok, I see. It's not a big problem to edit docs. :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6367][SQL] Use the proper data type for...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5094#issuecomment-86819376 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29275/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6367][SQL] Use the proper data type for...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5094#issuecomment-86819348 [Test build #29275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29275/consoleFull) for PR 5094 at commit [`e8b4599`](https://github.com/apache/spark/commit/e8b4599e780ea3dcb8dcadedb1fc61b1e4cde83b). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5218#issuecomment-86817620 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29273/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5218#issuecomment-86817610 [Test build #29273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29273/consoleFull) for PR 5218 at commit [`e2e30ce`](https://github.com/apache/spark/commit/e2e30ce2fa98d6aa8fcd05b12ca08de4064a05cd). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Minor] [SQL] Follow-up of PR #5210
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5219#issuecomment-86816367 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29282/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6526][ML] Add Normalizer transformer in...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5181#issuecomment-86816336 [Test build #29283 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29283/consoleFull) for PR 5181 at commit [`6faa7bf`](https://github.com/apache/spark/commit/6faa7bfdfd06d0b86fa1ad9e8d2c3722d2e90435). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Minor] [SQL] Follow-up of PR #5210
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5219#issuecomment-86816358 [Test build #29282 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29282/consoleFull) for PR 5219 at commit [`41f3a09`](https://github.com/apache/spark/commit/41f3a09f44112bf6e5553b6fee097ef4170433fd). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6544][build] Increment Avro version fro...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5193#issuecomment-86815673 Yeah LGTM - I think bumping maintenance releases like this has usually been fine for us with Avro. It may be best not merge this into 1.3 though, since we typically keep dependencies fixed in patch releases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Minor] [SQL] Follow-up of PR #5210
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5219#issuecomment-86813656 [Test build #29282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29282/consoleFull) for PR 5219 at commit [`41f3a09`](https://github.com/apache/spark/commit/41f3a09f44112bf6e5553b6fee097ef4170433fd). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Minor] [SQL] Follow-up of PR #5210
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5219#issuecomment-86812865 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6341][mllib] Upgrade breeze from 0.11.1...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5222#issuecomment-86812100 [Test build #29281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29281/consoleFull) for PR 5222 at commit [`ad8a688`](https://github.com/apache/spark/commit/ad8a688e980d7c2cbd5d16cbc146f7fb85934cbe). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5063#issuecomment-86812055 [Test build #29272 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29272/consoleFull) for PR 5063 at commit [`223aed9`](https://github.com/apache/spark/commit/223aed924c19c3862e2f1b29d9c8c98d3fc86e54). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5063#issuecomment-86812076 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29272/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6560][CORE] Do not suppress exceptions ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5223#issuecomment-86811925 [Test build #29280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29280/consoleFull) for PR 5223 at commit [`f42e92d`](https://github.com/apache/spark/commit/f42e92de46967e0b8dc19c198ea0ea6196c6ebca). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6560][CORE] Do not suppress exceptions ...
GitHub user stephenh opened a pull request: https://github.com/apache/spark/pull/5223 [SPARK-6560][CORE] Do not suppress exceptions from writer.write. If there is a failure in the Hadoop backend while calling writer.write, we should remember this original exception, and try to call writer.close(), but if that fails as well, still report the original exception. Note that, if writer.write fails, it is likely that writer was left in an invalid state, and so actually makes it more likely that writer.close will also fail. Which just increases the chances for writer.write's exception to be suppressed. This patch introduces an admittedly potentially too cute Utils.tryWithSafeFinally method to handle the try/finally gyrations. You can merge this pull request into a Git repository by running: $ git pull https://github.com/stephenh/spark do_not_suppress_writer_exception Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/5223.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5223 commit f42e92de46967e0b8dc19c198ea0ea6196c6ebca Author: Stephen Haberman Date: 2015-03-27T04:01:28Z [SPARK-6560][CORE] Do not suppress exceptions from writer.write. If there is a failure in the Hadoop backend while calling writer.write, we should remember this original exception, and try to call writer.close(), but if that fails as well, still report the original exception. Note that, if writer.write fails, it is likely that writer was left in an invalid state, and so actually makes it more likely that writer.close will also fail. Which just increases the chances for writer.write's exception to be suppressed. This patch introduces an admittedly potentially too cute Utils.tryWithSafeFinally method to handle the try/finally gyrations. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6341][mllib] Upgrade breeze from 0.11.1...
GitHub user yu-iskw opened a pull request: https://github.com/apache/spark/pull/5222 [SPARK-6341][mllib] Upgrade breeze from 0.11.1 to 0.11.2 There are any bugs of breeze's SparseVector at 0.11.1. You know, Spark 1.3 depends on breeze 0.11.1. So I think we should upgrade it to 0.11.2. https://issues.apache.org/jira/browse/SPARK-6341 And thanks you for your great cooperation, David Hall(@dlwh) You can merge this pull request into a Git repository by running: $ git pull https://github.com/yu-iskw/spark upgrade-breeze Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/5222.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5222 commit ad8a688e980d7c2cbd5d16cbc146f7fb85934cbe Author: Yu ISHIKAWA Date: 2015-03-27T04:04:01Z Upgrade breeze from 0.11.1 to 0.11.2 because of a bug of SparseVector. Thanks you for your great cooperation, David Hall(@dlwh) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6549 - Spark console logger logs to stde...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5202#issuecomment-86810509 Changing the default logging behavior like this would break compatibility for people who rely on the out-of-the-box behavior. IIRC this change was proposed previously and rejected on the same grounds. Also, I'm not even convinced that stdout is the better default. The reason it's stderr is to isolate user output from the spark's own output. In any case, I think on compatibility grounds we can't merge this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/5063#discussion_r27274938 --- Diff: docs/running-on-mesos.md --- @@ -211,6 +211,14 @@ See the [configuration page](configuration.html) for information on Spark config + spark.mesos.executor.cores + 1.0 + +The setting, which can be a floating point number, controls the number of cores allocated --- End diff -- Sorry, maybe I should have been more clear, but I meant for this text to go in *in addition* to the paragraph you originally had above it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6113] [mllib] [WIP] Stabilize DecisionT...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5009#issuecomment-86809663 [Test build #29279 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29279/consoleFull) for PR 5009 at commit [`7534133`](https://github.com/apache/spark/commit/75341330e79e6f8f96f33469a92f700f187e5eaa). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6113] [mllib] [WIP] Stabilize DecisionT...
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5009#issuecomment-86809275 Final stretch! I just need to update the docs now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2475][Streaming] Check whether has enou...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5212#issuecomment-86808633 [Test build #29278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29278/consoleFull) for PR 5212 at commit [`197a030`](https://github.com/apache/spark/commit/197a0304c70ef5262e30065fbec12595c1c72cb1). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/5218#issuecomment-86807495 LGTM pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6556][Core] Fix wrong parsing logic of ...
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/5209#issuecomment-86807463 Had some nits on the comments. Otherwise this LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5218#issuecomment-86807377 [Test build #29277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29277/consoleFull) for PR 5218 at commit [`1d6d1be`](https://github.com/apache/spark/commit/1d6d1be9ef335c51b477d8c2f661a72cd1c82f73). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6556][Core] Fix wrong parsing logic of ...
Github user sryza commented on a diff in the pull request:
https://github.com/apache/spark/pull/5209#discussion_r27274404
--- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ---
@@ -49,12 +49,17 @@ private[spark] class HeartbeatReceiver(sc:
SparkContext, scheduler: TaskSchedule
// executor ID -> timestamp of when the last heartbeat from this
executor was received
private val executorLastSeen = new mutable.HashMap[String, Long]
-
- private val executorTimeoutMs = sc.conf.getLong("spark.network.timeout",
-sc.conf.getLong("spark.storage.blockManagerSlaveTimeoutMs", 120)) *
1000
-
- private val checkTimeoutIntervalMs =
sc.conf.getLong("spark.network.timeoutInterval",
-sc.conf.getLong("spark.storage.blockManagerTimeoutIntervalMs", 60)) *
1000
+
+ // `spark.network.timeout` use `seconds`,
--- End diff --
Minor nits on the comments:
* I wouldn't use backticks here because it's not md-formatted.
* "use" -> "uses"
* "while" should be on line above
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/5218#discussion_r27274352
--- Diff:
core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala ---
@@ -49,10 +49,20 @@ class KryoSerializer(conf: SparkConf)
with Logging
with Serializable {
- private val bufferSize =
-(conf.getDouble("spark.kryoserializer.buffer.mb", 0.064) * 1024 *
1024).toInt
+ private val bufferSizeMb =
conf.getDouble("spark.kryoserializer.buffer.mb", 0.064)
+ if (bufferSizeMb >= 2048) {
+throw new IllegalArgumentException("spark.kryoserializer.buffer.mb
must be less than " +
+ s"2048 mb, got: + $bufferSizeMb mb.")
+ }
+ private val bufferSize = (bufferSizeMb * 1024 * 1024).toInt
+
+ val maxBufferSizeMb = conf.getInt("spark.kryoserializer.buffer.max.mb",
64)
+ if (maxBufferSizeMb >= 2048) {
+throw new IllegalArgumentException("spark.kryoserializer.buffer.max.mb
must be less than " +
+ s"2048 mb, got: + $maxBufferSizeMb mb.")
+ }
+ private val maxBufferSize = maxBufferSizeMb * 1026 * 1024
--- End diff --
good catch =D
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6559][PySpark]specify port of python ga...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5221#issuecomment-86807129 [Test build #29276 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29276/consoleFull) for PR 5221 at commit [`1e5c85d`](https://github.com/apache/spark/commit/1e5c85d7480c6ca91381c8c1629653d4943fc638). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user sryza commented on a diff in the pull request:
https://github.com/apache/spark/pull/5218#discussion_r27274319
--- Diff:
core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala ---
@@ -49,10 +49,20 @@ class KryoSerializer(conf: SparkConf)
with Logging
with Serializable {
- private val bufferSize =
-(conf.getDouble("spark.kryoserializer.buffer.mb", 0.064) * 1024 *
1024).toInt
+ private val bufferSizeMb =
conf.getDouble("spark.kryoserializer.buffer.mb", 0.064)
+ if (bufferSizeMb >= 2048) {
+throw new IllegalArgumentException("spark.kryoserializer.buffer.mb
must be less than " +
+ s"2048 mb, got: + $bufferSizeMb mb.")
+ }
+ private val bufferSize = (bufferSizeMb * 1024 * 1024).toInt
+
+ val maxBufferSizeMb = conf.getInt("spark.kryoserializer.buffer.max.mb",
64)
+ if (maxBufferSizeMb >= 2048) {
+throw new IllegalArgumentException("spark.kryoserializer.buffer.max.mb
must be less than " +
+ s"2048 mb, got: + $maxBufferSizeMb mb.")
+ }
+ private val maxBufferSize = maxBufferSizeMb * 1026 * 1024
--- End diff --
Is this meant to be *1026* * 1024?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6559][PySpark]specify port of python ga...
GitHub user WangTaoTheTonic opened a pull request: https://github.com/apache/spark/pull/5221 [SPARK-6559][PySpark]specify port of python gateway server https://issues.apache.org/jira/browse/SPARK-6559 Add a config "spark.python.gateway.port" to specify gateway server's port, and use Utils.startServiceOnPort to give it a retry if bind failed. /cc @davies You can merge this pull request into a Git repository by running: $ git pull https://github.com/WangTaoTheTonic/spark SPARK-6559 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/5221.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5221 commit 1e5c85d7480c6ca91381c8c1629653d4943fc638 Author: WangTaoTheTonic Date: 2015-03-27T03:20:22Z specify port of python gateway server --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6367][SQL] Use the proper data type for...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5094#issuecomment-86806795 [Test build #29275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29275/consoleFull) for PR 5094 at commit [`e8b4599`](https://github.com/apache/spark/commit/e8b4599e780ea3dcb8dcadedb1fc61b1e4cde83b). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-86806544 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29270/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-86806537 [Test build #29270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29270/consoleFull) for PR 731 at commit [`193d687`](https://github.com/apache/spark/commit/193d6873a33cd8582c329e62d34c5fe52d5b6c1e). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5218#issuecomment-86805660 [Test build #29273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29273/consoleFull) for PR 5218 at commit [`e2e30ce`](https://github.com/apache/spark/commit/e2e30ce2fa98d6aa8fcd05b12ca08de4064a05cd). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6521][Core]executors in the same node r...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5178#issuecomment-86805460 [Test build #29274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29274/consoleFull) for PR 5178 at commit [`bb94736`](https://github.com/apache/spark/commit/bb9473641c575e5022b95ee89372fe8981aa0814). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2475][Streaming] Check whether has enou...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5212#issuecomment-86804756 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29271/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2475][Streaming] Check whether has enou...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5212#issuecomment-86804722 [Test build #29271 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29271/consoleFull) for PR 5212 at commit [`8cf26b1`](https://github.com/apache/spark/commit/8cf26b1a1a4b1e311077146e992f64cd701aa7b8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6145][SQL] fix ORDER BY on nested field...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/5189#issuecomment-86803347 Ah, so `SELECT a.b` is equal to `SELECT a.b AS b`, and the `b` in `ORDER BY b` should reference to `a.b`, that's how hive does resolution and makes sense. I have sent you a PR to remove the duplicated `resolveGetField` and others LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6405] Limiting the maximum Kryo buffer ...
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/5218#issuecomment-86803042 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-86798128 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29269/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-86798107 [Test build #29269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29269/consoleFull) for PR 731 at commit [`f74cd8e`](https://github.com/apache/spark/commit/f74cd8e29368add06e6fc960cf11988f25d5418e). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][Minor] typo
Github user chenghao-intel closed the pull request at: https://github.com/apache/spark/pull/5220 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5063#issuecomment-86796016 [Test build #29272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29272/consoleFull) for PR 5063 at commit [`223aed9`](https://github.com/apache/spark/commit/223aed924c19c3862e2f1b29d9c8c98d3fc86e54). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][Minor] typo
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/5220#issuecomment-86795864 Oh, thanks @jeanlyn , that's weird, github doesn't show conflict. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][Minor] typo
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/5220#issuecomment-86795793 Yeap it is already fixed. But it is weird that this PR did not make conflict or something. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/5063#issuecomment-86795209 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6526][ML] Add Normalizer transformer in...
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/5181#issuecomment-86792888 I'll shorten the code. Thanks for the comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6552][Deploy][Doc]expose start-slave.sh...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/5205#issuecomment-86792171 After taking a look at description of `SPARK_MASTER_IP/SPARK_MASTER_PORT` in `spark-env.sh.template` and `start-master.sh`, I think these two configs might be designed to be used for binding master process to a different ip/port on master node. The description in `spark-env.sh.template`: ># - SPARK_MASTER_IP, to bind the master to a different IP address or hostname # - SPARK_MASTER_PORT / SPARK_MASTER_WEBUI_PORT, to use non-default ports for the master @srowen How do you think? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6552][Deploy][Doc]expose start-slave.sh...
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/5205#issuecomment-86790610 I just thought that this create the chance to make the variables in spark-env.sh to be inconsistent with what are actually in usethough it might be just a minor issue --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6552][Deploy][Doc]expose start-slave.sh...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/5205#issuecomment-86790310 I don't think so. IIUC start-slaves.sh read SPARK_MASTER_PORT/IP to pass them to other slave nodes so that the workers on remote nodes can read it and connect to the master. Let start-slave to read master's ip&port from other source is kinda making the start process more complicated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][Minor] typo
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5220#issuecomment-86790057 [Test build #29267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29267/consoleFull) for PR 5220 at commit [`f468358`](https://github.com/apache/spark/commit/f4683580243e296f17962d5e12d8210c1515202f). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][Minor] typo
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5220#issuecomment-86790064 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29267/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6526][ML] Add Normalizer transformer in...
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5181#issuecomment-86789981 Thanks for adding the test suite! I added 2 small comments. My only other thought is that there is some code duplication in the test suite. If you can shorten the code by eliminating duplication, that would be great. Other than these items, I think it's good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4688#issuecomment-86789908 [Test build #29268 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29268/consoleFull) for PR 4688 at commit [`f74303c`](https://github.com/apache/spark/commit/f74303c89c8fe542122680077e2a471d41b809c2). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class AMDelegationTokenRenewer(sparkConf: SparkConf, hadoopConf: Configuration) extends Logging ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
