[GitHub] spark pull request: [SPARK-9031] Merge BlockObjectWriter and DiskB...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7391#issuecomment-121148012 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9031] Merge BlockObjectWriter and DiskB...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7391#issuecomment-121148004 [Test build #37209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37209/console) for PR 7391 at commit [`89dc797`](https://github.com/apache/spark/commit/89dc797f543f0e5bfcc21223f88c9cfd1e10d2ea). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/7392#discussion_r34541871
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala
---
@@ -70,17 +70,27 @@ object GenerateOrdering extends
CodeGenerator[Seq[SortOrder], Ordering[InternalR
"""
}.mkString("\n")
+val mutableStates = ctx.mutableStates.map {
+ case (jt, name, _) => s"private $jt $name;"
+}.mkString("\n ")
+
+val initStates = ctx.mutableStates.zipWithIndex.map {
+ case ((jt, name, _), index) => s"$name = (${ctx.boxedType(jt)})
states[$index];"
+}.mkString("\n")
+
val code = s"""
- public SpecificOrdering generate($exprType[] expr) {
-return new SpecificOrdering(expr);
+ public SpecificOrdering generate($exprType[] expr, Object[] states) {
--- End diff --
can't we pass in the code to initialize the variables, rather than using an
object array?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7392#issuecomment-121147857 [Test build #37213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37213/consoleFull) for PR 7392 at commit [`fd45c7a`](https://github.com/apache/spark/commit/fd45c7aa90670301779b56fe84e6f2d6afb191d1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7392#issuecomment-121147735 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4362] [MLLIB] Make prediction probabili...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/7376#issuecomment-121147720 The prediction is already checked in the existing tests. I can add a check for sum to 1 though, yes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7392#issuecomment-121147754 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Simplify localFraction in CoalescedRDDPartitio...
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/7384#discussion_r34541669 --- Diff: core/src/main/scala/org/apache/spark/rdd/CoalescedRDD.scala --- @@ -58,7 +58,7 @@ private[spark] case class CoalescedRDDPartition( val parentPreferredLocations = rdd.context.getPreferredLocs(rdd, p.index).map(_.host) preferredLocation.exists(parentPreferredLocations.contains) } -if (parents.size == 0) 0.0 else (loc.toDouble / parents.size.toDouble) --- End diff -- This isn't worth the overhead of bothering with review and commit. I'd close this PR please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/7392#discussion_r34541628
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
---
@@ -56,6 +56,14 @@ class CodeGenContext {
*/
val references: mutable.ArrayBuffer[Expression] = new
mutable.ArrayBuffer[Expression]()
+ /**
+ * Holding expressions' mutable states like `Rand.rng`, and keep them as
member variables
+ * in generated classes like `SpecificProjection`.
+ * Each element is a 3-tuple: java type, variable name, variable value.
+ */
+ val mutableStates: mutable.ArrayBuffer[(String, String, Any)] =
--- End diff --
let's add a function so we don;t call += directly on the array buffer.
```scala
def addMutableState(javaType: String, variableName: String, initialValue:
String): Unit
```
this way it also makes it more clear what the semantics are for each
element in the tuple3.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8761][Deploy]Added synchronization in r...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/7364#discussion_r34541643
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -727,40 +727,44 @@ private[master] class Master(
def removeApplication(app: ApplicationInfo, state:
ApplicationState.Value) {
if (apps.contains(app)) {
- logInfo("Removing app " + app.id)
- apps -= app
- idToApp -= app.id
- endpointToApp -= app.driver
- addressToApp -= app.driver.address
- if (completedApps.size >= RETAINED_APPLICATIONS) {
-val toRemove = math.max(RETAINED_APPLICATIONS / 10, 1)
-completedApps.take(toRemove).foreach( a => {
- appIdToUI.remove(a.id).foreach { ui => webUi.detachSparkUI(ui) }
- applicationMetricsSystem.removeSource(a.appSource)
-})
-completedApps.trimStart(toRemove)
- }
- completedApps += app // Remember it in our history
- waitingApps -= app
+ synchronized{
--- End diff --
It's a premature optimization -- why?
Also I don't think this really addresses the full problem, as there are
other methods modifying the same state concurrently. I'd rather not commit
something incomplete.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/7392#discussion_r34541583
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateProjection.scala
---
@@ -151,85 +151,96 @@ object GenerateProjection extends
CodeGenerator[Seq[Expression], Projection] {
s"""if (!nullBits[$i]) arr[$i] = c$i;"""
}.mkString("\n ")
+val mutableStates = ctx.mutableStates.map {
+ case (jt, name, _) => s"private $jt $name;"
+}.mkString("\n ")
+
+val initStates = ctx.mutableStates.zipWithIndex.map {
+ case ((jt, name, _), index) => s"$name = (${ctx.boxedType(jt)})
states[$index];"
+}.mkString("\n")
+
val code = s"""
-public SpecificProjection generate($exprType[] expr) {
- return new SpecificProjection(expr);
+public SpecificProjection generate($exprType[] expr, Object[] states) {
+ return new SpecificProjection(expr, states);
}
class SpecificProjection extends ${classOf[BaseProject].getName} {
private $exprType[] expressions = null;
+ $mutableStates
- public SpecificProjection($exprType[] expr) {
+ public SpecificProjection($exprType[] expr, Object[] states) {
expressions = expr;
+$initStates
}
@Override
public Object apply(Object r) {
-return new SpecificRow(expressions, (InternalRow) r);
+return new SpecificRow((InternalRow) r);
}
-}
-final class SpecificRow extends ${classOf[MutableRow].getName} {
+ final class SpecificRow extends ${classOf[MutableRow].getName} {
--- End diff --
I made `SpecificRow` a inner class of `SpecificProjection` here, so that we
can access these mutable states easily.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/7392#discussion_r34541509
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/expressions/MonotonicallyIncreasingID.scala
---
@@ -49,4 +55,17 @@ private[sql] case class MonotonicallyIncreasingID()
extends LeafExpression {
count += 1
(TaskContext.get().partitionId().toLong << 33) + currentCount
}
+
+ override def genCode(ctx: CodeGenContext, ev: GeneratedExpressionCode):
String = {
+val countTerm = ctx.freshName("count")
+val partitionMaskTerm = ctx.freshName("partitionMask")
+ctx.mutableStates += (("long", countTerm, count))
+ctx.mutableStates += (("long", partitionMaskTerm, partitionMask))
--- End diff --
i think you should just explicitly initialize partiitonmask, rather than
relying on the current value (i.e. generate the code to initialize it with task
context)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/7392#discussion_r34541483
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/expressions/MonotonicallyIncreasingID.scala
---
@@ -49,4 +55,17 @@ private[sql] case class MonotonicallyIncreasingID()
extends LeafExpression {
count += 1
(TaskContext.get().partitionId().toLong << 33) + currentCount
--- End diff --
use partitionMask here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Minor][SQL] Allow spaces in the beginning and...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/7390#discussion_r34541474
--- Diff: unsafe/src/main/java/org/apache/spark/unsafe/types/Interval.java
---
@@ -43,12 +43,13 @@
* Finally is the unit name, ends with an optional "s".
*/
private static String unitRegex(String unit) {
-return "(?:\\s+(-?\\d+)\\s+" + unit + "s?)?";
+return "(?:\\s+(-?\\d+)\\s+" + unit + "s?\\s*)?";
--- End diff --
Sort-of related question -- why is this class in 'unsafe', and why not use
Joda time? the JDK already has a rich set of abstractions for time. It seems
like it's only used in catalyst.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121147125 [Test build #9 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/9/console) for PR 6783 at commit [`0493dc4`](https://github.com/apache/spark/commit/0493dc4fadf97d148c4cd157efa7719bb1b9f8c0). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Pmod(left: Expression, right: Expression) extends BinaryArithmetic ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121147149 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121146721 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121146662 [Test build #8 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/8/console) for PR 7241 at commit [`2fc7291`](https://github.com/apache/spark/commit/2fc7291333e45979ff4eed3511008b752dc6bb69). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] More comprehensive type chec...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/7348#discussion_r34541318
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/AbstractDataType.scala
---
@@ -96,6 +96,24 @@ private[sql] class TypeCollection(private val types:
Seq[AbstractDataType])
private[sql] object TypeCollection {
+ /**
+ * Types that can be ordered/compared. In the long run we should
probably make this a trait
+ * that can be mixed into each data type, and perhaps create an
[[AbstractDataType]].
+ */
+ val Ordered = TypeCollection(
+BooleanType,
+ByteType, ShortType, IntegerType, LongType,
+FloatType, DoubleType, DecimalType,
+TimestampType, DateType,
+StringType)
+
+ /**
+ * Types that can be used in bitwise operations.
+ */
+ val Bitwise = TypeCollection(
+BooleanType,
--- End diff --
yes
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7392#issuecomment-121146644 [Test build #37212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37212/consoleFull) for PR 7392 at commit [`12a9e12`](https://github.com/apache/spark/commit/12a9e12059d8663dc9fdd647d942b32152858f2f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7392#issuecomment-121146453 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9015] [BUILD] Clean project import in s...
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/7375#discussion_r34541288 --- Diff: pom.xml --- @@ -319,17 +306,6 @@ 1.0.0 - --- End diff -- Note that removing this from the dependencies list will remove Groovy as a provided dependency from all of the Spark project POMs. I always thought it was really confusing how the published Maven artifacts would declare a dependency on Groovy, so I'm glad to see this cleaned up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7392#issuecomment-121146474 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/7392 [SPARK-9020][SQL] Support mutable state in code gen expressions You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloud-fan/spark mutable-state Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7392.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7392 commit 12a9e12059d8663dc9fdd647d942b32152858f2f Author: Wenchen Fan Date: 2015-07-14T06:45:33Z Support mutable state in code gen expressions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9020][SQL] Support mutable state in cod...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/7392#issuecomment-121146269 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9031] Merge BlockObjectWriter and DiskB...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7391#issuecomment-121146035 [Test build #37209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37209/consoleFull) for PR 7391 at commit [`89dc797`](https://github.com/apache/spark/commit/89dc797f543f0e5bfcc21223f88c9cfd1e10d2ea). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] More comprehensive type chec...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7348#issuecomment-121146059 [Test build #37210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37210/consoleFull) for PR 7348 at commit [`4932d57`](https://github.com/apache/spark/commit/4932d57b257078df8432824e6a8d9e3a09c4993e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121145972 [Test build #37211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37211/consoleFull) for PR 7241 at commit [`2fc7291`](https://github.com/apache/spark/commit/2fc7291333e45979ff4eed3511008b752dc6bb69). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] More comprehensive type chec...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7348#issuecomment-121145911 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] More comprehensive type chec...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/7348#discussion_r34541015
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/AbstractDataType.scala
---
@@ -96,6 +96,24 @@ private[sql] class TypeCollection(private val types:
Seq[AbstractDataType])
private[sql] object TypeCollection {
+ /**
+ * Types that can be ordered/compared. In the long run we should
probably make this a trait
+ * that can be mixed into each data type, and perhaps create an
[[AbstractDataType]].
+ */
+ val Ordered = TypeCollection(
+BooleanType,
+ByteType, ShortType, IntegerType, LongType,
+FloatType, DoubleType, DecimalType,
+TimestampType, DateType,
+StringType)
+
+ /**
+ * Types that can be used in bitwise operations.
+ */
+ val Bitwise = TypeCollection(
+BooleanType,
--- End diff --
Does boolean support bitwise?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] More comprehensive type chec...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7348#issuecomment-121145896 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121145916 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121145903 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9031] Merge BlockObjectWriter and DiskB...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7391#issuecomment-121145910 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] More comprehensive type chec...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/7348#discussion_r34541019
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/AbstractDataType.scala
---
@@ -96,6 +96,24 @@ private[sql] class TypeCollection(private val types:
Seq[AbstractDataType])
private[sql] object TypeCollection {
+ /**
+ * Types that can be ordered/compared. In the long run we should
probably make this a trait
+ * that can be mixed into each data type, and perhaps create an
[[AbstractDataType]].
+ */
+ val Ordered = TypeCollection(
+BooleanType,
+ByteType, ShortType, IntegerType, LongType,
+FloatType, DoubleType, DecimalType,
+TimestampType, DateType,
+StringType)
+
+ /**
+ * Types that can be used in bitwise operations.
+ */
+ val Bitwise = TypeCollection(
+BooleanType,
--- End diff --
Does boolean support bitwise?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9031] Merge BlockObjectWriter and DiskB...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7391#issuecomment-121145892 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121145780 Master's tests might be in a bad state right now. If all else fails, I can try manually running this tomorrow on my local machine to make sure that tests pass, then can merge and re-trigger tests on all Python PRs. Thanks for your patience while we've been fighting build / test break issues. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121145758 [Test build #37208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37208/consoleFull) for PR 6783 at commit [`0493dc4`](https://github.com/apache/spark/commit/0493dc4fadf97d148c4cd157efa7719bb1b9f8c0). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Minor][SQL] Allow spaces in the beginning and...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7390#issuecomment-121145726 [Test build #37207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37207/consoleFull) for PR 7390 at commit [`57861f7`](https://github.com/apache/spark/commit/57861f7726a91f733ff563b4d46e50a9eb5a8483). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] [WIP] Leverage ExpectsInputT...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7348#issuecomment-121145755 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121145699 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] [WIP] Leverage ExpectsInputT...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7348#issuecomment-121145739 [Test build #37202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37202/console) for PR 7348 at commit [`e4727cc`](https://github.com/apache/spark/commit/e4727cc538590112b9eb48bae77b8ec454227c2f). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class And(left: Expression, right: Expression) extends BinaryOperator with Predicate ` * `case class Or(left: Expression, right: Expression) extends BinaryOperator with Predicate ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9031] Merge BlockObjectWriter and DiskB...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7391#issuecomment-121145405 /cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9031] Merge BlockObjectWriter and DiskB...
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/7391 [SPARK-9031] Merge BlockObjectWriter and DiskBlockObject writer to remove abstract class BlockObjectWriter has only one concrete non-test class, DiskBlockObjectWriter. In order to simplify the code in preparation for other refactorings, I think that we should remove this base class and have only DiskBlockObjectWriter. While at one time we may have planned to have multiple BlockObjectWriter implementations, that doesn't seem to have happened, so the extra abstraction seems unnecessary. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JoshRosen/spark shuffle-write-interface-refactoring Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7391.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7391 commit 1607c913bc4c8de937029871a3cd942c37598533 Author: Josh Rosen Date: 2015-07-14T00:58:48Z Merge BlockObjectWriter and DiskBlockObjectWriter commit 5755918aefc51622e73dfb0c6741c28b70720923 Author: Josh Rosen Date: 2015-07-14T00:59:30Z Remove unnecessary val in case class commit 89dc797f543f0e5bfcc21223f88c9cfd1e10d2ea Author: Josh Rosen Date: 2015-07-14T06:36:12Z Rename test suite. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121145024 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121145035 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121144971 [Test build #11 has started](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/11/consoleFull) for PR 6783 at commit [`0493dc4`](https://github.com/apache/spark/commit/0493dc4fadf97d148c4cd157efa7719bb1b9f8c0). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Minor][SQL] Allow spaces in the beginning and...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7390#issuecomment-121145034 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9026] Refactor SimpleFutureAction.onCom...
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/7385#issuecomment-121145033 > @zsxwing will TaskContext.get() work for you? AFAIK we're trying to avoid the introduction of new withContext methods when TaskContext.get() will suffice (see some of the old discussions regarding mapPartitionsWithContext, etc.). Forgot it. It should work. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Minor][SQL] Allow spaces in the beginning and...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7390#issuecomment-121145017 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121144899 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121144936 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121144923 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9026] Refactor SimpleFutureAction.onCom...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7385#issuecomment-121144882 @zsxwing will `TaskContext.get()` work for you? AFAIK we're trying to avoid the introduction of new `withContext` methods when `TaskContext.get()` will suffice (see some of the old discussions regarding `mapPartitionsWithContext`, etc.). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121144873 [Test build #37196 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37196/console) for PR 7241 at commit [`2fc7291`](https://github.com/apache/spark/commit/2fc7291333e45979ff4eed3511008b752dc6bb69). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9026] Refactor SimpleFutureAction.onCom...
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/7385#issuecomment-121144704 @JoshRosen thank you for fixing this. Actually, I have another requirement for `submitJob`. Current `submitJob` signature is ``` def submitJob[T, U, R]( rdd: RDD[T], processPartition: Iterator[T] => U, partitions: Seq[Int], resultHandler: (Int, U) => Unit, resultFunc: => R): SimpleFutureAction[R] ``` I cannot access `TaskContext` in this method. Could you change `processPartition: Iterator[T] => U` to `processPartition: (TaskContext, Iterator[T]) => U`, or add a new method for it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-121144687 [Test build #37205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37205/console) for PR 5267 at commit [`080274b`](https://github.com/apache/spark/commit/080274b8a4ced53c758175e894c648d5e2318d59). * This patch **fails Scala style tests**. * This patch **does not merge cleanly**. * This patch adds the following public classes _(experimental)_: * `case class ClusterNodeStat (` * `class BisectingKMeansModel(val node: ClusterNode) extends Serializable with Logging ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-121144688 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Minor][SQL] Allow spaces in the beginning and...
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/7390 [Minor][SQL] Allow spaces in the beginning and ending of string for Interval This is a minor fixing for #7355 to allow spaces in the beginning and ending of string parsed to `Interval`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/viirya/spark-1 fix_interval_string Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7390.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7390 commit 815a9cb438ef99ad0640a70b39501db1d94887ed Author: Liang-Chi Hsieh Date: 2015-07-14T06:17:58Z Slightly modify regex to allow spaces in the beginning and ending of string. commit 57861f7726a91f733ff563b4d46e50a9eb5a8483 Author: Liang-Chi Hsieh Date: 2015-07-14T06:31:43Z Fix scala style. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user zhichao-li commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121144576 retest this please. [Fail by timeout after 1 minute](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37198/testReport/org.apache.spark.sql.hive.thriftserver/CliSuite/Single_command_with___database/) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-121144033 [Test build #37205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37205/consoleFull) for PR 5267 at commit [`080274b`](https://github.com/apache/spark/commit/080274b8a4ced53c758175e894c648d5e2318d59). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6284][MESOS] Add mesos role, principal ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4960#issuecomment-121144099 [Test build #37206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37206/consoleFull) for PR 4960 at commit [`32da933`](https://github.com/apache/spark/commit/32da9336607013c1be6b2273f7545174fce57e50). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8911] Fix local mode endless heartbeats
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7382#issuecomment-121143886 [Test build #37203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37203/consoleFull) for PR 7382 at commit [`1258bdf`](https://github.com/apache/spark/commit/1258bdf0c0a87ef6aa4eff73ad83bb15728b88d1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7292] Local checkpointing
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7279#issuecomment-121143509 [Test build #37204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37204/consoleFull) for PR 7279 at commit [`db70dc2`](https://github.com/apache/spark/commit/db70dc23262bc7baab8d4eacbbf6df0ef2187306). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] [WIP] Leverage ExpectsInputT...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/7348#discussion_r34540368
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala
---
@@ -48,6 +49,9 @@ case class UnaryMinus(child: Expression) extends
UnaryArithmetic {
case class UnaryPositive(child: Expression) extends UnaryArithmetic {
override def prettyName: String = "positive"
+ override def checkInputDataTypes(): TypeCheckResult =
+TypeUtils.checkForNumericExpr(child.dataType, "operator -")
--- End diff --
should be "operator +"?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6284][MESOS] Add mesos role, principal ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4960#issuecomment-121143349 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-121143332 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-121143339 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7292] Local checkpointing
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7279#issuecomment-121143340 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8911] Fix local mode endless heartbeats
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7382#issuecomment-121143335 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6284][MESOS] Add mesos role, principal ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4960#issuecomment-121143334 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7292] Local checkpointing
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7279#issuecomment-121143325 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8911] Fix local mode endless heartbeats
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7382#issuecomment-121143323 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...
Github user yu-iskw commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-121143278 @mengxr Sorry for the delay of my modification. I have just modified the implementation as follows. Could you review it when you have time? I am checking the performance on EC2. I'll get back to you later. - Remove Python API (will add later) - Rename the class name from HierarchicalClustering to BisectingKMeans - Change the way how to initialize the child centers - Use avg. cost for the criterion, not variance --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] [WIP] Leverage ExpectsInputT...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7348#issuecomment-121143164 [Test build #37202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37202/consoleFull) for PR 7348 at commit [`e4727cc`](https://github.com/apache/spark/commit/e4727cc538590112b9eb48bae77b8ec454227c2f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6284][MESOS] Add mesos role, principal ...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4960#issuecomment-121142791 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] [WIP] Leverage ExpectsInputT...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7348#issuecomment-121142497 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7292] Local checkpointing
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/7279#issuecomment-121142486 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8911] Fix local mode endless heartbeats
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/7382#issuecomment-121142551 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8993][SQL] [WIP] Leverage ExpectsInputT...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7348#issuecomment-121142433 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6851] [SQL] function least/greatest fol...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7387#issuecomment-121141423 [Test build #37201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37201/consoleFull) for PR 7387 at commit [`e8c2e09`](https://github.com/apache/spark/commit/e8c2e09505916d981a64827728ef305719d58834). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6851] [SQL] function least/greatest fol...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7387#issuecomment-121141229 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6851] [SQL] function least/greatest fol...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7387#issuecomment-121141206 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6851] [SQL] function least/greatest fol...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7387#issuecomment-121141117 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6851] [SQL] function least/greatest fol...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7387#issuecomment-121141101 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6851] [SQL] function least/greatest fol...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7387#issuecomment-121141144 [Test build #10 has started](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/10/consoleFull) for PR 7387 at commit [`e8c2e09`](https://github.com/apache/spark/commit/e8c2e09505916d981a64827728ef305719d58834). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6851] [SQL] function least/greatest fol...
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/7387#issuecomment-121141030 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4362] [MLLIB] Make prediction probabili...
Github user acidghost closed the pull request at: https://github.com/apache/spark/pull/6761 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4362] [MLLIB] Make prediction probabili...
Github user acidghost commented on the pull request: https://github.com/apache/spark/pull/7376#issuecomment-121140598 Looks good to me. There's no check that they sum to 1 or that the prediction from probabilities is right, but I don't think is necessary --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121140048 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8221][SQL]Add pmod function
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6783#issuecomment-121140029 [Test build #37198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37198/console) for PR 6783 at commit [`0493dc4`](https://github.com/apache/spark/commit/0493dc4fadf97d148c4cd157efa7719bb1b9f8c0). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Pmod(left: Expression, right: Expression) extends BinaryArithmetic ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8962] Add Scalastyle rule to ban direct...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7350#issuecomment-121139541 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8962] Add Scalastyle rule to ban direct...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7350#issuecomment-121139518 [Test build #37190 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37190/console) for PR 7350 at commit [`e3e96f7`](https://github.com/apache/spark/commit/e3e96f705f968f234837b8f14e464d4916014b9b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9026] Refactor SimpleFutureAction.onCom...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7385#issuecomment-121139415 [Test build #37200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37200/consoleFull) for PR 7385 at commit [`1e2db7f`](https://github.com/apache/spark/commit/1e2db7f54952aa9338acea6e2b64ca24d61ed2ab). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9026] Refactor SimpleFutureAction.onCom...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7385#issuecomment-121139114 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9026] Refactor SimpleFutureAction.onCom...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7385#issuecomment-121139099 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6851] [SQL] function least/greatest fol...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7387#issuecomment-121139087 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6851] [SQL] function least/greatest fol...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7387#issuecomment-121139039 [Test build #37195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37195/console) for PR 7387 at commit [`e8c2e09`](https://github.com/apache/spark/commit/e8c2e09505916d981a64827728ef305719d58834). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Least(children: Seq[Expression]) extends Expression ` * `case class Greatest(children: Seq[Expression]) extends Expression ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8840][SparkR] Add float coercion on Spa...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7280#issuecomment-121138135 [Test build #37199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37199/consoleFull) for PR 7280 at commit [`dbf0c1b`](https://github.com/apache/spark/commit/dbf0c1bf6d0234ee5df55ca9040430e9ab3da872). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8840][SparkR] Add float coercion on Spa...
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/7280#issuecomment-121137898 @shivaram @davies I updated this to implicitly convert double to float based on schema. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
