GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/16550
[SPARK-19178][SQL] convert string of large numbers to int should return null
## What changes were proposed in this pull request?
When we convert a string to integral, we will convert that string to
`decimal(20, 0)` first, so that we can turn a string with decimal format to
truncated integral, e.g. `CAST('1.2' AS int)` will return `1`.
However, this brings problems when we convert a string with large numbers
to integral, e.g. `CAST('1234567890123' AS int)` will return `1912276171`,
while Hive returns null as we expected.
This is a long standing bug(seems it was there the first day Spark SQL was
created), this PR fixes this bug by adding the native support to convert
`UTF8String` to integral.
## How was this patch tested?
new regression tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark string-to-int
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16550.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16550
----
commit afa7066b394217316789c06db467acf40c74cd28
Author: Wenchen Fan <wenc...@databricks.com>
Date: 2017-01-11T08:14:44Z
native support for converting UTF8String to integral
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
