GitHub user DonnyZone opened a pull request:
https://github.com/apache/spark/pull/18960
[SPARK-21739][SQL]Cast expression should initialize timezoneId when it is
called statically to convert something into TimestampType
## What changes were proposed in this pull request?
https://issues.apache.org/jira/projects/SPARK/issues/SPARK-21739
This issue is caused by introducing TimeZoneAwareExpression.
When the **Cast** expression converts something into TimestampType, it
should be resolved with setting `timezoneId`. In general, it is resolved in
LogicalPlan phase.
However, there are still some places that use Cast expression statically to
convert datatypes without setting `timezoneId`. In such cases,
`NoSuchElementException: None.get` will be thrown for TimestampType.
This PR is proposed to fix the issue. We have checked the whole project and
found two such usages(i.e., in`TableReader` and `HiveTableScanExec`).
## How was this patch tested?
unit test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DonnyZone/spark spark-21739
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18960.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18960
----
commit 2537c1ea5028541a68ae63e8f5eea8eb8f62dadf
Author: donnyzone <[email protected]>
Date: 2017-08-16T07:53:19Z
spark-21739
commit 86331e37550f4204c8c4ee2c409fbd6000654f43
Author: donnyzone <[email protected]>
Date: 2017-08-16T07:55:17Z
fix code style
commit 492b756fde5008854d1351ed423c3897c683c662
Author: donnyzone <[email protected]>
Date: 2017-08-16T07:56:34Z
correct answer
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
