GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/21824
[SPARK-24871][SQL] Refactor Concat and MapConcat to avoid creating
concatenator object for each row.
## What changes were proposed in this pull request?
Refactor `Concat` and `MapConcat` to:
- avoid creating concatenator object for each row.
- make `Concat` handle `containsNull` properly.
- make `Concat` shortcut if `null` child is found.
## How was this patch tested?
Added some tests and existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ueshin/apache-spark
issues/SPARK-24871/refactor_concat_mapconcat
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21824.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21824
----
commit c4d4b28756d93ecc5c1e9a50e705e2a9cb4b8460
Author: Takuya UESHIN <ueshin@...>
Date: 2018-07-19T08:32:15Z
Refactor `Concat` and `MapConcat` to avoid creating concatenator for each
row.
commit 644a30d32e390f39f25515bb251e120e69502f27
Author: Takuya UESHIN <ueshin@...>
Date: 2018-07-19T10:38:20Z
Make `Concat` shortcut if null child is found.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
