GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/22749
[WIP][SPARK-25746][SQL] Refactoring ExpressionEncoder to get rid of flat flag ## What changes were proposed in this pull request? This is inspired during implementing #21732. For now `ScalaReflection` needs to consider how `ExpressionEncoder` uses generated serializers and deserializers. And `ExpressionEncoder` has a weird `flat` flag. After discussion with @cloud-fan, it seems to be better to refactor `ExpressionEncoder`. It should make SPARK-24762 easier to do. ## How was this patch tested? Existing tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/viirya/spark-1 SPARK-24762-refactor Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22749.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22749 ---- commit e1b5deebe715479125c8878f0c90a55dc9ab3e85 Author: Liang-Chi Hsieh <viirya@...> Date: 2018-07-09T03:42:04Z Aggregator should be able to use Option of Product encoder. commit 80506f4e98184ccd66dbaac14ec52d69c358020d Author: Liang-Chi Hsieh <viirya@...> Date: 2018-07-13T04:40:55Z Enable top-level Option of Product encoders. commit ed3d5cb697b10af2e2cf4c78ab521d4d0b2f3c9b Author: Liang-Chi Hsieh <viirya@...> Date: 2018-08-24T04:26:28Z Remove topLevel parameter. commit 9fc3f6165156051142a8366a32726badaaa16bb7 Author: Liang-Chi Hsieh <viirya@...> Date: 2018-08-24T04:37:39Z Merge remote-tracking branch 'upstream/master' into SPARK-24762 commit 5f95bd0cf1bd308c7df55c41caef7a9f19368f5d Author: Liang-Chi Hsieh <viirya@...> Date: 2018-08-24T04:42:33Z Remove useless change. commit a4f04055b2ba22f371663565710328791942855a Author: Liang-Chi Hsieh <viirya@...> Date: 2018-08-24T14:38:16Z Add more tests. commit c1f798f7e9cba0d04223eed06f1b1f547ec29dc5 Author: Liang-Chi Hsieh <viirya@...> Date: 2018-08-25T01:52:01Z Add test. commit 80e11d289d7775863cb9c28b2c1d4364292048a4 Author: Liang-Chi Hsieh <viirya@...> Date: 2018-10-06T04:06:57Z Merge remote-tracking branch 'upstream/master' into SPARK-24762 commit 0f029b0a28700334dc6334f1ad89b3124f235a51 Author: Liang-Chi Hsieh <viirya@...> Date: 2018-10-06T04:40:07Z Improve code comments. commit d755e8406f06117ccc96b8f19debab6b2a736e10 Author: Liang-Chi Hsieh <viirya@...> Date: 2018-10-15T09:55:03Z Refactoring ExpressionEncoder. ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org