[GitHub] spark pull request #13581: [SPARK-14321][SQL] Reduce date format cost and st...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/13581


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13581: [SPARK-14321][SQL] Reduce date format cost and st...

[cloud-fan]

Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/13581#discussion_r66527669
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
 ---
@@ -434,24 +436,21 @@ abstract class UnixTime extends BinaryExpression with 
ExpectsInputTypes {
 left.dataType match {
   case StringType if right.foldable =>
 val sdf = classOf[SimpleDateFormat].getName
-val fString = if (constFormat == null) null else 
constFormat.toString
-val formatter = ctx.freshName("formatter")
-if (fString == null) {
+if (formatter == null) {
   ev.copy(code = s"""
 boolean ${ev.isNull} = true;
 ${ctx.javaType(dataType)} ${ev.value} = 
${ctx.defaultValue(dataType)};""")
 } else {
+  val formatterName = ctx.addReferenceObj("formatter", formatter, 
sdf)
--- End diff --

nit: maybe we can just use `formatter` as name


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13581: [SPARK-14321][SQL] Reduce date format cost and st...

[cloud-fan]

Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/13581#discussion_r66525197
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
 ---
@@ -549,21 +552,22 @@ case class FromUnixTime(sec: Expression, format: 
Expression)
   override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
 val sdf = classOf[SimpleDateFormat].getName
 if (format.foldable) {
-  if (constFormat == null) {
+  if (formatter == null) {
 ev.copy(code = s"""
--- End diff --

same here


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13581: [SPARK-14321][SQL] Reduce date format cost and st...

[cloud-fan]

Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/13581#discussion_r66525010
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
 ---
@@ -434,24 +436,21 @@ abstract class UnixTime extends BinaryExpression with 
ExpectsInputTypes {
 left.dataType match {
   case StringType if right.foldable =>
 val sdf = classOf[SimpleDateFormat].getName
-val fString = if (constFormat == null) null else 
constFormat.toString
-val formatter = ctx.freshName("formatter")
-if (fString == null) {
+if (formatter == null) {
   ev.copy(code = s"""
--- End diff --

this can be: `ev.copy(code = "", isNull = "false", value = 
ctx.defaultValue(dataType))`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #13581: [SPARK-14321][SQL] Reduce date format cost and st...

[hvanhovell]

GitHub user hvanhovell opened a pull request:

https://github.com/apache/spark/pull/13581

[SPARK-14321][SQL] Reduce date format cost and string-to-date cost in date 
functions

## What changes were proposed in this pull request?
The current implementations of `UnixTime` and `FromUnixTime` do not cache 
their parser/formatter as much as they could. This PR resolved this issue.

This PR is a take over from https://github.com/apache/spark/pull/13522 and 
further optimizes the re-use of the parser/formatter. It also fixes the 
improves handling (catching the actual exception instead of `Throwable`). All 
credits for this work should go to @rajeshbalamohan.

## How was this patch tested?
Current tests.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hvanhovell/spark SPARK-14321

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/13581.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #13581


commit 602d4a70ba845df3160a07c2c9afe2d5c3c574c4
Author: Rajesh Balamohan 
Date:   2016-06-06T12:54:02Z

[SPARK-14321][SQL] Reduce date format cost and string-to-date cost in date 
functions

commit 425aa7ea30f1f92143fc9f539c7b928255dee1c9
Author: Rajesh Balamohan 
Date:   2016-06-07T08:17:20Z

[SPARK-14321][SQL] Reduce date format cost and string-to-date cost in date 
functions

commit a0861285ccffe362a1a5758f9c99a36e6f7b2f5d
Author: Herman van Hovell 
Date:   2016-06-09T21:07:49Z

Polishing code gen and exceptions




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org