Repository: spark Updated Branches: refs/heads/branch-1.0 96fdc7c38 -> 16c8d562d
[SPARK-2509][SQL] Add optimization for Substring. `Substring` including `null` literal cases could be added to `NullPropagation`. Author: Takuya UESHIN <ues...@happy-camper.st> Closes #1428 from ueshin/issues/SPARK-2509 and squashes the following commits: d9eb85f [Takuya UESHIN] Add Substring cases to NullPropagation. (cherry picked from commit 9b38b7c71352bb5e6d359515111ad9ca33299127) Signed-off-by: Reynold Xin <r...@apache.org> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/16c8d562 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/16c8d562 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/16c8d562 Branch: refs/heads/branch-1.0 Commit: 16c8d562d403b2a8b9ac85b27081df9074d9f002 Parents: 96fdc7c Author: Takuya UESHIN <ues...@happy-camper.st> Authored: Tue Jul 15 22:35:34 2014 -0700 Committer: Reynold Xin <r...@apache.org> Committed: Tue Jul 15 22:35:43 2014 -0700 ---------------------------------------------------------------------- .../scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala | 3 +++ 1 file changed, 3 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/16c8d562/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ---------------------------------------------------------------------- diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala index 042a56d..9d28f41 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala @@ -171,6 +171,9 @@ object NullPropagation extends Rule[LogicalPlan] { case Literal(candidate, _) if candidate == v => true case _ => false })) => Literal(true, BooleanType) + case e @ Substring(Literal(null, _), _, _) => Literal(null, e.dataType) + case e @ Substring(_, Literal(null, _), _) => Literal(null, e.dataType) + case e @ Substring(_, _, Literal(null, _)) => Literal(null, e.dataType) // Put exceptional cases above if any case e: BinaryArithmetic => e.children match { case Literal(null, _) :: right :: Nil => Literal(null, e.dataType)
