GitHub user MaxGekk opened a pull request:
https://github.com/apache/spark/pull/21578
[SPARK-24571][SQL] Support Char literals
## What changes were proposed in this pull request?
In the PR, I propose to automatically convert a `Literal` with `Char` type
to a `Literal` of `String` type. Currently, the following code:
```scala
val df = Seq("Amsterdam", "San Francisco", "London").toDF("city")
df.where($"city".contains('o')).show(false)
```
fails with the exception:
```
Unsupported literal type class java.lang.Character o
java.lang.RuntimeException: Unsupported literal type class
java.lang.Character o
at
org.apache.spark.sql.catalyst.expressions.Literal$.apply(literals.scala:78)
```
The PR fixes this issue by converting `char` to `string` of length `1`. I
believe it makes sense to does not differentiate `char` and `string(1)` in _a
unified, multi-language data platform_ like Spark which supports languages like
Python.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/MaxGekk/spark-1 support-char-literals
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21578.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21578
----
commit 97097f58866a492e9bb450de53c5f51ec8ef78e6
Author: Maxim Gekk <maxim.gekk@...>
Date: 2018-06-13T22:29:12Z
A test for converting Char to String
commit 87640c768f9c1c71defb2f476eabbfea67e0f9ed
Author: Maxim Gekk <maxim.gekk@...>
Date: 2018-06-13T23:20:39Z
Support Char in StringConverter
commit 0fb4669f47a5abcfdf255d6fd99f070764343cfc
Author: Maxim Gekk <maxim.gekk@...>
Date: 2018-06-13T23:21:43Z
Evaluate Char literal as String literal
commit 99dfbfe5fc299efe2ece9bea9493dcfe1d5b523d
Author: Maxim Gekk <maxim.gekk@...>
Date: 2018-06-14T03:03:56Z
Added a test for filtering rows by using Char literal
commit 56fd59254d4483b7b9ff866cec7f0cf87c28bb14
Author: Maxim Gekk <maxim.gekk@...>
Date: 2018-06-14T03:05:56Z
Cover the case of java.lang.Character
commit 657f7be117f6b96500c8e2d7d764bf96d94efa8f
Author: Maxim Gekk <maxim.gekk@...>
Date: 2018-06-14T03:27:26Z
Improving of the test
commit 0f09ab2a1ea0584ae61e5f44fd604882208d956c
Author: Maxim Gekk <maxim.gekk@...>
Date: 2018-06-16T03:31:05Z
Merge remote-tracking branch 'origin/master' into char-to-string
commit 42101465583b11665317cba714a778ba5e8404fe
Author: Maxim Gekk <maxim.gekk@...>
Date: 2018-06-16T03:47:10Z
Adding ticket number to test's titles
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
