GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/12322
[SPARK-14554][SQL] Dataset.map may generate wrong java code for wide table ## What changes were proposed in this pull request? In https://github.com/apache/spark/pull/12047/files#diff-94a1f59bcc9b6758c4ca874652437634R529, we may split field expressions codes in `CreateExternalRow` to support wide table. However, after we support whole stage codegen for typed operations, the `CreateExternalRow` may also appear in whole stage codegen. The problem is, in whole stage codegen, the input for expressions is not always the input row, but can be `CodeGenContext.currentVars`, which doesn't work well with `CodeGenContext.splitExpressions`. This PR stops splitting field expressions codes in `CreateExternalRow` if it's being whole stage codegened, but we need a better way to fix this problem instead of avoiding it. ## How was this patch tested? new test in DatasetSuite. You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloud-fan/spark codegen Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/12322.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #12322 ---- commit 0ad119490ac5fba06d6cdeda1d4a9e9e0f8c0c8e Author: Wenchen Fan <wenc...@databricks.com> Date: 2016-04-12T03:43:08Z Dataset.map may generate wrong java code for wide table ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org