GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/11596
[SPARK-12719][SQL] [WIP] SQL generation support for generators, including UDTF ## What changes were proposed in this pull request? This is a alternate way to convert SQL from analyzed logical plans containing Generate operator. In this PR , generators in projection list are expressed as LATERAL VIEW. Sample Plan : ``` GlobalLimit 3 +- LocalLimit 3 +- Project [gencol2#204] +- Generate explode(gencol1#203), true, false, Some(gentab2), [gencol2#204] +- Generate explode(array(array(1, 2, 3))), true, false, Some(gentab1), [gencol1#203] +- MetastoreRelation default, t4, None ``` Generated Query: ``` SELECT `gentab2`.`gencol2` FROM `default`.`t4` LATERAL VIEW explode(array(array(1, 2, 3))) `gentab1` AS `gencol1` LATERAL VIEW explode(`gentab1`.`gencol1`) `gentab2` AS `gencol2` LIMIT 3 ``` ## How was this patch tested? Tests added to LogicalPlanToSQLSuite (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) You can merge this pull request into a Git repository by running: $ git pull https://github.com/dilipbiswal/spark SPARK-12719-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/11596.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #11596 ---- commit fe8b0642994fd976681d7e675712738bb6f7076b Author: Dilip Biswal <dbis...@us.ibm.com> Date: 2016-03-05T09:49:58Z [SPARK-13698] Fix Analysis Exceptions when Using Backticks in Generate commit 1e9b683febba74192d5a882bcd38d0a47dd630bf Author: Dilip Biswal <dbis...@us.ibm.com> Date: 2016-03-07T07:42:00Z [SPARK-12719] SQL generation support for generators (including UDTF) commit 865f5a5afaaff2c1eb101630328c362dc63e0ddf Author: Dilip Biswal <dbis...@us.ibm.com> Date: 2016-03-07T21:28:48Z Code review comments commit c037d8ccc008bcba518a4be62da385d1fd037756 Author: Dilip Biswal <dbis...@us.ibm.com> Date: 2016-03-08T02:36:21Z Review comments from Reynold commit e10883a720c7e10d5a5294093c473ff6ff358d0c Author: Dilip Biswal <dbis...@us.ibm.com> Date: 2016-03-08T07:35:46Z Alternative approach. ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org