fqaiser94 commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-646429505
> Can you explain more about this? Please see the comment from @dbtsai [here](https://github.com/apache/spark/pull/27066#issuecomment-609512144) for a more thorough explanation. TBH I'm uncertain now if reusing `CreateNamedStruct` will give us access to any further optimizations beyond what the optimization rules I've added in this PR will do for `WithFields` anyway. > My last concern is we convert WithField to CreateNamedStruct in a kind of random place in the optimizer. I'm thinking about something like .... Interesting idea but I'm not sure if it's feasible. I tried code like this: ``` case class WithFields( structExpr: Expression, nameExprs: Seq[Expression], valExprs: Seq[Expression]) extends Expression { ... override def eval(input: InternalRow = null): Any = toCreateNamedStruct.eval(input) override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = toCreateNamedStruct.doGenCode(ctx, ev) } ``` I get the following compile time error: ``` [error] Access to protected method doGenCode not permitted because [error] prefix type org.apache.spark.sql.catalyst.expressions.Expression does not conform to [error] class WithFields in package expressions where the access takes place [error] toCreateNamedStruct.doGenCode(ctx, ev) ``` Please let me know if there's a trick around this. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
