[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-167792794 It is ok. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-167715428 oh sorry I haven't noticed it... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/9883 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-167714278 Looks like I need to close another pr again... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158931041 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158931045 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46527/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158930708 **[Test build #46527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46527/consoleFull)** for PR 9883 at commit [`e8d4b10`](https://github.com/apache/spark/commit/e8d4b108c6d829841db045c6a626f1eb1bd80ee7). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `case class Hash(children: Expression*) extends Expression with ImplicitCastInputTypes `\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158905853 **[Test build #46527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46527/consoleFull)** for PR 9883 at commit [`e8d4b10`](https://github.com/apache/spark/commit/e8d4b108c6d829841db045c6a626f1eb1bd80ee7). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158900491 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158900494 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46518/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158900367 **[Test build #46518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46518/consoleFull)** for PR 9883 at commit [`d9d28a3`](https://github.com/apache/spark/commit/d9d28a36a7e6da35f456a47cb35adc7720afd51a). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158879311 **[Test build #46518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46518/consoleFull)** for PR 9883 at commit [`d9d28a3`](https://github.com/apache/spark/commit/d9d28a36a7e6da35f456a47cb35adc7720afd51a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158859505 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158859506 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46509/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158859349 **[Test build #46509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46509/consoleFull)** for PR 9883 at commit [`afbbcf2`](https://github.com/apache/spark/commit/afbbcf2d5819810f8047507d71158dea5edce22e). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `case class Hash(children: Expression*) extends Expression with ImplicitCastInputTypes `\n * `abstract class Aggregator[-I, B, O] extends Serializable `\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158851369 **[Test build #46509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46509/consoleFull)** for PR 9883 at commit [`afbbcf2`](https://github.com/apache/spark/commit/afbbcf2d5819810f8047507d71158dea5edce22e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/9883#discussion_r45571886
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala ---
@@ -424,16 +424,34 @@ trait Row extends Serializable {
true
}
- override def hashCode: Int = {
-// Using Scala's Seq hash code implementation.
-var n = 0
-var h = MurmurHash3.seqSeed
-val len = length
-while (n < len) {
- h = MurmurHash3.mix(h, apply(n).##)
- n += 1
-}
-MurmurHash3.finalizeHash(h, n)
+ override def hashCode: Int = hashCode(this)
+
+ def hashCode(v: Any): Int = v match {
+case null => 0
+case b: Boolean => if (b) 1 else 0
+case b: Byte => b.toInt
+case s: Short => s.toInt
+case i: Int => i
+case l: Long => (l ^ (l >>> 32)).toInt
+case f: Float => java.lang.Float.floatToIntBits(f)
+case d: Double =>
+ val b = java.lang.Double.doubleToLongBits(d)
+ (b ^ (b >>> 32)).toInt
+case a: Array[Byte] => java.util.Arrays.hashCode(a)
+case s: String => s.getBytes.foldLeft(0) { (acc, n) => acc * 31 + n }
+case a: Array[_] => a.foldLeft(0) { (acc, n) => acc * 31 + hashCode(n)
}
+case s: Seq[_] => s.foldLeft(0) { (acc, n) => acc * 31 + hashCode(n) }
+case m: Map[_, _] =>
+ var r = 0
+ m.foreach { case (k, v) => r += hashCode(k) ^ hashCode(v) }
--- End diff --
Because we only add the `hashCode`s of keys and values of a map. Their
iterated order should be no problem.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/9883#discussion_r45571868
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
---
@@ -45,6 +48,66 @@ case class Md5(child: Expression) extends
UnaryExpression with ImplicitCastInput
}
}
+case class Hash(children: Expression*) extends Expression with
ImplicitCastInputTypes {
+
+ override def inputTypes: Seq[AbstractDataType] = children.map(_.dataType)
+ override def dataType: DataType = IntegerType
+
+ override def nullable: Boolean = children.exists(_.nullable)
+
+ @transient
+ private lazy val extractProjection =
GenerateSafeProjection.generate(children)
+
+ lazy val schema = StructType(children.zipWithIndex.map { case (e, idx) =>
+StructField(s"_c$idx", e.dataType)
+ })
+ @transient
+ private lazy val encoder = RowEncoder(schema)
+
+ def getProjection: Projection = extractProjection
+ def getEncoder: ExpressionEncoder[Row] = encoder
+
+ override def eval(input: InternalRow): Any = {
+val internalRow: InternalRow = extractProjection(input)
--- End diff --
We can't directly use the `hashCode` of `InternalRow`. Because in
`InternalRow` we store data as internal types instead of its original types.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/9883#discussion_r45571748
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala ---
@@ -424,16 +424,34 @@ trait Row extends Serializable {
true
}
- override def hashCode: Int = {
-// Using Scala's Seq hash code implementation.
-var n = 0
-var h = MurmurHash3.seqSeed
-val len = length
-while (n < len) {
- h = MurmurHash3.mix(h, apply(n).##)
- n += 1
-}
-MurmurHash3.finalizeHash(h, n)
+ override def hashCode: Int = hashCode(this)
+
+ def hashCode(v: Any): Int = v match {
+case null => 0
+case b: Boolean => if (b) 1 else 0
+case b: Byte => b.toInt
+case s: Short => s.toInt
+case i: Int => i
+case l: Long => (l ^ (l >>> 32)).toInt
+case f: Float => java.lang.Float.floatToIntBits(f)
+case d: Double =>
+ val b = java.lang.Double.doubleToLongBits(d)
+ (b ^ (b >>> 32)).toInt
+case a: Array[Byte] => java.util.Arrays.hashCode(a)
+case s: String => s.getBytes.foldLeft(0) { (acc, n) => acc * 31 + n }
+case a: Array[_] => a.foldLeft(0) { (acc, n) => acc * 31 + hashCode(n)
}
+case s: Seq[_] => s.foldLeft(0) { (acc, n) => acc * 31 + hashCode(n) }
+case m: Map[_, _] =>
+ var r = 0
+ m.foreach { case (k, v) => r += hashCode(k) ^ hashCode(v) }
+ r
+case r: Row =>
--- End diff --
These hashCode implementations are following Hive's implementations.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158846758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158846740 **[Test build #46502 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46502/consoleFull)** for PR 9883 at commit [`47072b8`](https://github.com/apache/spark/commit/47072b8dcb0a6676a9d143bb4a46df014f3b9a10). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `case class Hash(children: Expression*) extends Expression with ImplicitCastInputTypes `\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158846761 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46502/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158827710 **[Test build #46502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46502/consoleFull)** for PR 9883 at commit [`47072b8`](https://github.com/apache/spark/commit/47072b8dcb0a6676a9d143bb4a46df014f3b9a10). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158826613 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158775545 **[Test build #46488 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46488/consoleFull)** for PR 9883 at commit [`47072b8`](https://github.com/apache/spark/commit/47072b8dcb0a6676a9d143bb4a46df014f3b9a10). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `case class Hash(children: Expression*) extends Expression with ImplicitCastInputTypes `\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158775557 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46488/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158775556 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158762259 **[Test build #46488 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46488/consoleFull)** for PR 9883 at commit [`47072b8`](https://github.com/apache/spark/commit/47072b8dcb0a6676a9d143bb4a46df014f3b9a10). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158649259 Thanks. I will update this later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158649170 Yeah, the point appears to be to match Hive's, so it has to be documented and tested as such. Otherwise a generic unspecified 'hash' function doesn't help much. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158641892 Hmm, do we need to make the hash function result consistent with Hive's? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158625782 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46475/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158625781 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158625768 **[Test build #46475 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46475/consoleFull)** for PR 9883 at commit [`afe626c`](https://github.com/apache/spark/commit/afe626158691ec500a0653a4907d281044bb). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158613140 **[Test build #46475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46475/consoleFull)** for PR 9883 at commit [`afe626c`](https://github.com/apache/spark/commit/afe626158691ec500a0653a4907d281044bb). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/9883 [SPARK-8233][SQL] Add misc function hash JIRA: https://issues.apache.org/jira/browse/SPARK-8233 Add hash function that returns a hash value of the arguments. You can merge this pull request into a Git repository by running: $ git pull https://github.com/viirya/spark-1 misc-hash Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/9883.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #9883 commit afe626158691ec500a0653a4907d281044bb Author: Liang-Chi Hsieh Date: 2015-11-21T08:28:55Z Add misc function hash. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
