[GitHub] spark pull request: [SPARK-7955][Core] Ensure executors with cache...
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/6508#discussion_r31378538 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -150,6 +158,9 @@ private[spark] class ExecutorAllocationManager( // Metric source for ExecutorAllocationManager to expose internal status to MetricsSystem. val executorAllocationManagerSource = new ExecutorAllocationManagerSource + private lazy val sparkEnv = SparkEnv.get + + private val executorEndpoints = new mutable.HashMap[String, RpcEndpointRef]() --- End diff -- need new line after this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7962][MESOS] Fix master url parsing in ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6517 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7576] [MLLIB] Add spark.ml user guide d...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6501 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7962][MESOS] Fix master url parsing in ...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/6517#issuecomment-106998154 This looks fine as a hot fix, since mesos cluster mode is totally broken. In the future we should remove all standalone master references from the abstract class, since it really shouldn't belong there. Merging into master and 1.4. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7576] [MLLIB] Add spark.ml user guide d...
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/6501#issuecomment-106997805 LGTM, I'll merge with master and branch-1.4 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7918] [MLlib] MLlib Python doc parity c...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6461#issuecomment-106997731 [Test build #33803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33803/consoleFull) for PR 6461 at commit [`940e3f1`](https://github.com/apache/spark/commit/940e3f13d7da070e9bed19c07960fc48102453dc). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7918] [MLlib] MLlib Python doc parity c...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6461#issuecomment-106997709 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7918] [MLlib] MLlib Python doc parity c...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6461#issuecomment-106997715 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106997413 [Test build #33802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33802/consoleFull) for PR 6515 at commit [`46ef7e5`](https://github.com/apache/spark/commit/46ef7e599c930b7fd5cc0881f99f254ab90cf722). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106997001 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106997021 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106996096 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user yijieshen commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106994994 I think so, would keep an eye on this and create lint-java if possible later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106994699 Ah ok. So the only way to run this is to use mvn compile? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][minor] remove unnecessary type coercion ...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/6516#discussion_r31378410
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala
---
@@ -511,32 +510,6 @@ trait HiveTypeCoercion {
}
/**
- * Casts to/from [[BooleanType]] are transformed into comparisons since
- * the JVM does not consider Booleans to be numeric types.
- */
- object BooleanCasts extends Rule[LogicalPlan] {
-def apply(plan: LogicalPlan): LogicalPlan = plan
transformAllExpressions {
- // Skip nodes who's children have not been resolved yet.
- case e if !e.childrenResolved => e
- // Skip if the type is boolean type already. Note that this extra
cast should be removed
- // by optimizer.SimplifyCasts.
- case Cast(e, BooleanType) if e.dataType == BooleanType => e
--- End diff --
i.e. this is basically constant folding.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][minor] remove unnecessary type coercion ...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/6516#discussion_r31378408
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala
---
@@ -511,32 +510,6 @@ trait HiveTypeCoercion {
}
/**
- * Casts to/from [[BooleanType]] are transformed into comparisons since
- * the JVM does not consider Booleans to be numeric types.
- */
- object BooleanCasts extends Rule[LogicalPlan] {
-def apply(plan: LogicalPlan): LogicalPlan = plan
transformAllExpressions {
- // Skip nodes who's children have not been resolved yet.
- case e if !e.childrenResolved => e
- // Skip if the type is boolean type already. Note that this extra
cast should be removed
- // by optimizer.SimplifyCasts.
- case Cast(e, BooleanType) if e.dataType == BooleanType => e
--- End diff --
I think this is an optimization, not for correctness?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user yijieshen commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106994629 Not that sure how lint-java works. If I understand correctly, in lint-java, we should run error prone separately like what lint-scala does? If so, seems not possible this moment, since error prone can only tightly coupled with maven's `compile` and `test-compile`, refer to issue: https://github.com/google/error-prone/issues/337 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7952][SQL] equality check between boole...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6505#issuecomment-106994617 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3873] [build] Add style checker to enfo...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6502#issuecomment-106994572 @vanzin - thanks, this looks great. Can you do 3 things? 1. Make the import ordering configurable. 2. The style rule is pretty complicated. Add some unit tests for it. 3. Submit a pull request against scalastyle proper. The reason is we should avoid one-offs that make it harder to upgrade in the future. We can merge this as soon as there is a pull request against scalastyle and way to move forward. And once that is merged into scalastyle and scalastyle releases a new version, we can remove our one-off rule. This is what we have done for all the one-off rules in the past. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106994447 The tests will still fail as we need 2 more changes to satisfy the type equal constraint for `EqualTo`. * https://github.com/apache/spark/pull/6505 to avoid something like `EqualTo(true, 2)` * https://github.com/apache/spark/pull/6516 to avoid something like `EqualTo(timestamp, Literal(0))` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106994335 [Test build #33801 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33801/consoleFull) for PR 6405 at commit [`89aa1d4`](https://github.com/apache/spark/commit/89aa1d42fbf54955dcd45740fd2cba7e2a396da8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106994241 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106994236 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106994189 Got it - as long as it is fine on Jenkins, I think it is OK. We can rely on Jenkins to do this anyway. One thing to add is lint-java file in /dev folder. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user yijieshen commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106993985 Test more Error bug patterns manually, work as expected. However, after an error report, if no `mvn clean` called before next `mvn package`, and not changing the buggy code, `maven-compiler-plugin` would just think all classes are up to date and not report the error again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993862 [Test build #33797 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33797/consoleFull) for PR 6405 at commit [`7e144e1`](https://github.com/apache/spark/commit/7e144e1d4adbd7ed023b89c77ed50003bc0952c0). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class TypeCheckResult(val errorMessage: String) extends AnyVal ` * `abstract class UnaryArithmetic extends UnaryExpression ` * `case class UnaryMinus(child: Expression) extends UnaryArithmetic ` * `case class Sqrt(child: Expression) extends UnaryArithmetic ` * `case class Abs(child: Expression) extends UnaryArithmetic ` * `case class BitwiseNot(child: Expression) extends UnaryArithmetic ` * `case class MaxOf(left: Expression, right: Expression) extends BinaryArithmetic ` * `case class MinOf(left: Expression, right: Expression) extends BinaryArithmetic ` * `case class Atan2(left: Expression, right: Expression)` * `case class Hypot(left: Expression, right: Expression)` * `case class EqualTo(left: Expression, right: Expression) extends BinaryComparison ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993873 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993749 [Test build #33799 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33799/consoleFull) for PR 6405 at commit [`9daa906`](https://github.com/apache/spark/commit/9daa90688e2324d1aeac5dcd28e8d8d888f447a2). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `abstract class UnaryArithmetic extends UnaryExpression ` * `case class UnaryMinus(child: Expression) extends UnaryArithmetic ` * `case class Sqrt(child: Expression) extends UnaryArithmetic ` * `case class Abs(child: Expression) extends UnaryArithmetic ` * `case class BitwiseNot(child: Expression) extends UnaryArithmetic ` * `case class MaxOf(left: Expression, right: Expression) extends BinaryArithmetic ` * `case class MinOf(left: Expression, right: Expression) extends BinaryArithmetic ` * `case class Atan2(left: Expression, right: Expression)` * `case class Hypot(left: Expression, right: Expression)` * `case class EqualTo(left: Expression, right: Expression) extends BinaryComparison ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993751 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7962][MESOS] Fix master url parsing in ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6517#issuecomment-106993746 [Test build #33800 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33800/consoleFull) for PR 6517 at commit [`61a1198`](https://github.com/apache/spark/commit/61a1198716d73b7226aaaee918b59f2dc06a6591). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993697 [Test build #33799 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33799/consoleFull) for PR 6405 at commit [`9daa906`](https://github.com/apache/spark/commit/9daa90688e2324d1aeac5dcd28e8d8d888f447a2). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993621 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7962][MESOS] Fix master url parsing in ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6517#issuecomment-106993629 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7962][MESOS] Fix master url parsing in ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6517#issuecomment-106993620 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993628 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7962][MESOS] Fix master url parsing in ...
GitHub user tnachen opened a pull request: https://github.com/apache/spark/pull/6517 [SPARK-7962][MESOS] Fix master url parsing in rest submission client. You can merge this pull request into a Git repository by running: $ git pull https://github.com/tnachen/spark fix_mesos_client Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/6517.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6517 commit 61a1198716d73b7226aaaee918b59f2dc06a6591 Author: Timothy Chen Date: 2015-05-30T06:03:49Z Fix master url parsing in rest submission client. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7962][MESOS] Fix master url parsing in ...
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/6517#issuecomment-106993604 @andrewor14 PTAL, this is against master branch --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993581 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993580 [Test build #33798 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33798/consoleFull) for PR 6405 at commit [`0515cf3`](https://github.com/apache/spark/commit/0515cf37be18039f96aebc008c814027c0c7db2c). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `abstract class UnaryArithmetic extends UnaryExpression ` * `case class UnaryMinus(child: Expression) extends UnaryArithmetic ` * `case class Sqrt(child: Expression) extends UnaryArithmetic ` * `case class Abs(child: Expression) extends UnaryArithmetic ` * `case class BitwiseNot(child: Expression) extends UnaryArithmetic ` * `case class MaxOf(left: Expression, right: Expression) extends BinaryArithmetic ` * `case class MinOf(left: Expression, right: Expression) extends BinaryArithmetic ` * `case class Atan2(left: Expression, right: Expression)` * `case class Hypot(left: Expression, right: Expression)` * `case class EqualTo(left: Expression, right: Expression) extends BinaryComparison ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993495 [Test build #33798 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33798/consoleFull) for PR 6405 at commit [`0515cf3`](https://github.com/apache/spark/commit/0515cf37be18039f96aebc008c814027c0c7db2c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [MESOS] Fix master url prefix stripping in spa...
Github user tnachen closed the pull request at: https://github.com/apache/spark/pull/6512 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993384 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106993379 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/6405#discussion_r31378244
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
---
@@ -86,12 +86,18 @@ abstract class Expression extends TreeNode[Expression] {
case (i1, i2) => i1 == i2
}
}
+
+ /**
+ * Check the input data types, returns `TypeCheckResult.success` if it's
valid,
+ * or return a `TypeCheckResult` with an error message if invalid.
+ */
+ def checkInputDataTypes(): TypeCheckResult = TypeCheckResult.success
--- End diff --
Sure sounds good to do this in a follow up pr. Mark a TODO here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7558] Guard against direct uses of FunS...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6510#issuecomment-106992823 Merged in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7558] Guard against direct uses of FunS...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6510 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7558] Guard against direct uses of FunS...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6510#issuecomment-106992727 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/6405#discussion_r31378221
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
---
@@ -86,12 +86,18 @@ abstract class Expression extends TreeNode[Expression] {
case (i1, i2) => i1 == i2
}
}
+
+ /**
+ * Check the input data types, returns `TypeCheckResult.success` if it's
valid,
+ * or return a `TypeCheckResult` with an error message if invalid.
+ */
+ def checkInputDataTypes(): TypeCheckResult = TypeCheckResult.success
--- End diff --
Then we need to implement it for a lot of expressions and decide error
message for them in this PR... Can we do it step by step and finally remove the
default implementation?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7705][Yarn] Cleanup of .sparkStaging di...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6409#issuecomment-106992586 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7705][Yarn] Cleanup of .sparkStaging di...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6409#issuecomment-106992582 [Test build #33795 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33795/consoleFull) for PR 6409 at commit [`aed9f18`](https://github.com/apache/spark/commit/aed9f18388e24d327ffdae5104331538dcd8e9bf). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106992479 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106992531 [Test build #33797 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33797/consoleFull) for PR 6405 at commit [`7e144e1`](https://github.com/apache/spark/commit/7e144e1d4adbd7ed023b89c77ed50003bc0952c0). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106992485 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106992204 No need to test all of them. As long as one works ... :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user yijieshen commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106992110 @rxin, I have insert [DeadException](http://errorprone.info/bugpattern/DeadException) `new Exception()`which is considered as ERROR in error prone, it reports correctly. Will test all the ERROR bug patterns manually to make sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7885][SQL]add config to control map agg...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6426#issuecomment-106992097 That patch is in this pull request: https://github.com/apache/spark/pull/5542 And jira ticket: https://issues.apache.org/jira/browse/SPARK-4233 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106991739 Thanks. Can you manually inject a "bug" into some Java code to see if it reports that correctly? If those are all the warnings, I'd just update the code to remove the warnings, and then turn warnings on as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7952][SQL] equality check between boole...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6505#issuecomment-106991600 [Test build #33794 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33794/consoleFull) for PR 6505 at commit [`18211e7`](https://github.com/apache/spark/commit/18211e700074a81cac2e60857a710375c8d952fc). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7952][SQL] equality check between boole...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6505#issuecomment-106991604 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][minor] remove unnecessary type coercion ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6516#issuecomment-106991411 [Test build #33796 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33796/consoleFull) for PR 6516 at commit [`d5035a4`](https://github.com/apache/spark/commit/d5035a459753873edae381ca0d68c402ac100c56). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/6405#discussion_r31378052 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -441,8 +406,7 @@ case class CaseKeyWhen(key: Expression, branches: Seq[Expression]) extends CaseW override def children: Seq[Expression] = key +: branches - override lazy val resolved: Boolean = -childrenResolved && valueTypesEqual + override protected def checkTypesInternal(): TypeCheckResult = TypeCheckResult.success --- End diff -- do we add the cast elsewhere? if we do, then i think it is ok to not check here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][minor] remove unnecessary type coercion ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6516#issuecomment-106991284 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][minor] remove unnecessary type coercion ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6516#issuecomment-106991275 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/6405#discussion_r31378039
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
---
@@ -86,12 +86,18 @@ abstract class Expression extends TreeNode[Expression] {
case (i1, i2) => i1 == i2
}
}
+
+ /**
+ * Check the input data types, returns `TypeCheckResult.success` if it's
valid,
+ * or return a `TypeCheckResult` with an error message if invalid.
+ */
+ def checkInputDataTypes(): TypeCheckResult = TypeCheckResult.success
--- End diff --
can we remove the default implementation?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/6405#discussion_r31378030
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCheckResult.scala
---
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.analysis
+
+/**
+ * Represents the result of `Expression.checkInputDataTypes`.
+ * We will throw `AnalysisException` in `CheckAnalysis` if error message
is not null.
+ *
+ */
+class TypeCheckResult(val errorMessage: String) extends AnyVal {
--- End diff --
mark the constructor as private to force callers to use the
object.success/fail factory methods. also update the documentation to say "Use
[[TypeCheckResult.success]] and [[TypeCheckResult.fail]] to instantiate this."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][minor] remove unnecessary type coercion ...
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/6516 [SQL][minor] remove unnecessary type coercion rule We have defined these logics in `Cast` already, I think we should remove this rule. You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloud-fan/spark tmp2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/6516.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6516 commit d5035a459753873edae381ca0d68c402ac100c56 Author: Wenchen Fan Date: 2015-05-30T05:20:26Z remove useless rule --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106991031 I chatted with @mateiz offline. It seems OK performance wise to cast. But we should look into what the standard thing is to do here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7957] Preserve partitioning when using ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6509 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user yijieshen commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106990886 @rxin: /Users/yijie/code/spark/network/common/src/test/java/org/apache/spark/network/RequestTimeoutIntegrationSuite.java:114: warning: [WaitNotInLoop] Object.wait() should always be called in a loop callback0.wait(FOREVER); ^ (see http://errorprone.info/bugpattern/WaitNotInLoop) /Users/yijie/code/spark/network/common/src/test/java/org/apache/spark/network/RequestTimeoutIntegrationSuite.java:122: warning: [WaitNotInLoop] Object.wait() should always be called in a loop callback1.wait(4 * 1000); ^ (see http://errorprone.info/bugpattern/WaitNotInLoop) /Users/yijie/code/spark/network/common/src/test/java/org/apache/spark/network/RequestTimeoutIntegrationSuite.java:162: warning: [WaitNotInLoop] Object.wait() should always be called in a loop callback0.wait(FOREVER); ^ (see http://errorprone.info/bugpattern/WaitNotInLoop) /Users/yijie/code/spark/network/common/src/test/java/org/apache/spark/network/RequestTimeoutIntegrationSuite.java:174: warning: [WaitNotInLoop] Object.wait() should always be called in a loop callback1.wait(FOREVER); ^ (see http://errorprone.info/bugpattern/WaitNotInLoop) /Users/yijie/code/spark/network/common/src/test/java/org/apache/spark/network/RequestTimeoutIntegrationSuite.java:222: warning: [WaitNotInLoop] Object.wait() should always be called in a loop callback0.wait(2 * 1000); ^ (see http://errorprone.info/bugpattern/WaitNotInLoop) /Users/yijie/code/spark/network/common/src/test/java/org/apache/spark/network/sasl/SparkSaslSuite.java:277: warning: [WaitNotInLoop] Object.wait() should always be called in a loop lock.wait(10 * 1000); ^ (see http://errorprone.info/bugpattern/WaitNotInLoop) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7957] Preserve partitioning when using ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6509#issuecomment-106990781 [Test build #33793 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33793/consoleFull) for PR 6509 at commit [`497465d`](https://github.com/apache/spark/commit/497465daaeec962b3f266c2c54c53d0f9d64d4e8). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7957] Preserve partitioning when using ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6509#issuecomment-106990784 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user yijieshen commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106990693 @rxin, I've tried locally, when compiler's warning are suppressed by default, nothing error occurs. When I turned on the warnings, it only warns on spurious wakeups of a test class, I would paste them here later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106990571 Jenkins, ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106990453 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6515#issuecomment-106990363 Thanks, @yijieshen. Have you tried running this locally and see what was produced? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7938][BUILD]Use Google ErrorProne durin...
GitHub user yijieshen opened a pull request: https://github.com/apache/spark/pull/6515 [SPARK-7938][BUILD]Use Google ErrorProne during Maven build of Spark [Error Prone](http://errorprone.info) augment the compilerâs type analysis and hooks into standard build, aiming at detecting Java code mistakes immediately after they are made. This PR hooks ErrorProne into maven-compiler-plugin to enable code analysis at maven's compile & test-compile phases. You can merge this pull request into a Git repository by running: $ git pull https://github.com/yijieshen/spark errorprone Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/6515.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6515 commit 46ef7e599c930b7fd5cc0881f99f254ab90cf722 Author: Yijie Shen Date: 2015-05-30T04:49:13Z Use Google ErrorProne into Maven build of Spark --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7576] [MLLIB] Add spark.ml user guide d...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6501#issuecomment-106990190 [Test build #33791 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33791/consoleFull) for PR 6501 at commit [`4ad93d5`](https://github.com/apache/spark/commit/4ad93d5533a2c66ce12e82b014da5b9907e8d4cc). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7576] [MLLIB] Add spark.ml user guide d...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6501#issuecomment-106990195 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [ML] [MLlib] [Docs] Updating ML Doc "Estimator...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6514#issuecomment-106990111 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [ML] [MLlib] [Docs] Updating ML Doc "Estimator...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6514#issuecomment-106990107 [Test build #33792 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33792/consoleFull) for PR 6514 at commit [`7d34939`](https://github.com/apache/spark/commit/7d349396a9cc4768615f42edc60ef9fce55623fe). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class SparkSinkSuite extends FunSuite ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7705][Yarn] Cleanup of .sparkStaging di...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6409#issuecomment-106983832 [Test build #33795 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33795/consoleFull) for PR 6409 at commit [`aed9f18`](https://github.com/apache/spark/commit/aed9f18388e24d327ffdae5104331538dcd8e9bf). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7705][Yarn] Cleanup of .sparkStaging di...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6409#issuecomment-106983633 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7705][Yarn] Cleanup of .sparkStaging di...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6409#issuecomment-106983628 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7459] [MLLIB] ElementwiseProduct Java e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6008#issuecomment-106983100 [Test build #33789 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33789/consoleFull) for PR 6008 at commit [`72e6dc0`](https://github.com/apache/spark/commit/72e6dc05290370b9eb3df041f7e6b38ea7a0c150). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7459] [MLLIB] ElementwiseProduct Java e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6008#issuecomment-106983114 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7558] Guard against direct uses of FunS...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6510#issuecomment-106982738 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7558] Guard against direct uses of FunS...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6510#issuecomment-106982734 [Test build #33790 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33790/consoleFull) for PR 6510 at commit [`6618b46`](https://github.com/apache/spark/commit/6618b46f33ba25adc42b6e439cb0af7cc5fa42c7). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7562][SPARK-6444][SQL] Improve error re...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6405#issuecomment-106982659 cc @mateiz since he wrote some of the decimal code. @mateiz - can you take a look at @cloud-fan's comment on whether we can always turn decimal comparison into Decimal.unlimited? My thought is that it might be too expensive (since it'd remove your decimal -> long optimization). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7952][SQL] equality check between boole...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6505#issuecomment-106982350 [Test build #33794 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33794/consoleFull) for PR 6505 at commit [`18211e7`](https://github.com/apache/spark/commit/18211e700074a81cac2e60857a710375c8d952fc). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7952][SQL] equality check between boole...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6505#issuecomment-106982264 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7952][SQL] equality check between boole...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6505#issuecomment-106982259 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [DOCS][Tiny] Added a missing dash(-) in docs/c...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6513 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [DOCS][Tiny] Added a missing dash(-) in docs/c...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6513#issuecomment-106982021 Thanks. I've merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7957] Preserve partitioning when using ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6509#issuecomment-106981513 [Test build #33793 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33793/consoleFull) for PR 6509 at commit [`497465d`](https://github.com/apache/spark/commit/497465daaeec962b3f266c2c54c53d0f9d64d4e8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7957] Preserve partitioning when using ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6509#issuecomment-106981416 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7957] Preserve partitioning when using ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6509#issuecomment-106981424 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][MLLIB][SPARK-4675][SPARK-4823]RowSimilar...
Github user rezazadeh commented on the pull request: https://github.com/apache/spark/pull/6213#issuecomment-106980262 Hi @debasish83 thank you for this PR. As it stands, it has too many components, which it makes it hard to review individual contributions. @mengxr and I spoke about this, and are wondering if you'd like to split it up to smaller PRs. In order, the PRs would be the following: 1) Adding rowSimilarities() for *just* cosine similarity (more similarity types adds extra reviewing, so please leave those out). Once this is done, then: 2) Adding similarProducts and similarUsers to MatrixFactorizationModel, once this is done, then: 3) Adding different similarity kernels Could you please close this PR and submit the above in order, one at a time? We should work on each in order, i.e. wait for one to be merged before the next one is started to be reviewed. The relevant JIRAs are 1) SPARK-4823, 2) SPARK-4675, and 3) is new. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6964][SQL] Support Cancellation in the ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6207#issuecomment-106980245 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6964][SQL] Support Cancellation in the ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6207#issuecomment-106980241 [Test build #33788 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33788/consoleFull) for PR 6207 at commit [`37bd362`](https://github.com/apache/spark/commit/37bd362f0282c1e1539436ca7e47ece19cb287f8). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7576] [MLLIB] Add spark.ml user guide d...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6501#issuecomment-106980114 [Test build #33791 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33791/consoleFull) for PR 6501 at commit [`4ad93d5`](https://github.com/apache/spark/commit/4ad93d5533a2c66ce12e82b014da5b9907e8d4cc). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
