[GitHub] spark pull request #22317: [SPARK-25310][SQL] ArraysOverlap may throw a Comp...

2018-09-03 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/22317


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22317: [SPARK-25310][SQL] ArraysOverlap may throw a Comp...

2018-09-03 Thread mgaido91 
Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/spark/pull/22317#discussion_r214646275
  
--- Diff: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala
 ---
@@ -471,9 +473,12 @@ class CollectionExpressionsSuite extends SparkFunSuite 
with ExpressionEvalHelper
   ArrayType(BinaryType))
 val b2 = Literal.create(Seq[Array[Byte]](Array[Byte](2, 1), 
Array[Byte](4, 3)),
   ArrayType(BinaryType))
+val b3 = Literal.create(Seq[Array[Byte]](Array[Byte](1, 2), 
Array[Byte](3, 4)),
+  ArrayType(BinaryType, false))
--- End diff --

nit: `ArrayType(BinaryType, containsNull = false)`


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22317: [SPARK-25310][SQL] ArraysOverlap may throw a Comp...

2018-09-03 Thread mgaido91 
Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/spark/pull/22317#discussion_r214646362
  
--- Diff: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala
 ---
@@ -439,6 +439,7 @@ class CollectionExpressionsSuite extends SparkFunSuite 
with ExpressionEvalHelper
 val a4 = Literal.create(Seq[String](null, ""), ArrayType(StringType))
 val a5 = Literal.create(Seq[String]("", "abc"), ArrayType(StringType))
 val a6 = Literal.create(Seq[String]("def", "ghi"), 
ArrayType(StringType))
+val a7 = Literal.create(Seq(1, 2, 3), ArrayType(IntegerType, false))
--- End diff --

ditto


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22317: [SPARK-25310][SQL] ArraysOverlap may throw a Comp...

2018-09-03 Thread mgaido91 
Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/spark/pull/22317#discussion_r214600395
  
--- Diff: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala
 ---
@@ -454,6 +455,8 @@ class CollectionExpressionsSuite extends SparkFunSuite 
with ExpressionEvalHelper
 checkEvaluation(ArraysOverlap(a4, a6), null)
 checkEvaluation(ArraysOverlap(a5, a6), false)
 
--- End diff --

nit: extra newline


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22317: [SPARK-25310][SQL] ArraysOverlap may throw a Comp...

2018-09-02 Thread kiszk 
Github user kiszk commented on a diff in the pull request:

https://github.com/apache/spark/pull/22317#discussion_r214559736
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
 ---
@@ -1623,12 +1623,13 @@ case class ArraysOverlap(left: Expression, right: 
Expression)
 val set = ctx.freshName("set")
 val addToSetFromSmallerCode = nullSafeElementCodegen(
   smaller, i, s"$set.add($getFromSmaller);", s"${ev.isNull} = true;")
+val setIsNullCode = if (nullable) s"${ev.isNull} = false;" else ""
 val elementIsInSetCode = nullSafeElementCodegen(
--- End diff --

Good catch, done.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22317: [SPARK-25310][SQL] ArraysOverlap may throw a Comp...

2018-09-02 Thread viirya 
Github user viirya commented on a diff in the pull request:

https://github.com/apache/spark/pull/22317#discussion_r214556466
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
 ---
@@ -1623,12 +1623,13 @@ case class ArraysOverlap(left: Expression, right: 
Expression)
 val set = ctx.freshName("set")
 val addToSetFromSmallerCode = nullSafeElementCodegen(
   smaller, i, s"$set.add($getFromSmaller);", s"${ev.isNull} = true;")
+val setIsNullCode = if (nullable) s"${ev.isNull} = false;" else ""
 val elementIsInSetCode = nullSafeElementCodegen(
--- End diff --

There are calls to `nullSafeElementCodegen` in `bruteForceCodegen`, for 
example, we don't need to fix them?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org