GideonPotok commented on PR #46597: URL: https://github.com/apache/spark/pull/46597#issuecomment-2119251289
> How about we go with this: > > > > - try to get the modified `Mode.update` approach working, and see how it does on the benchmark > > - limit the support to collated StringType only, no complex types for now - that can be a separate change > > > > hint: to properly limit the support for collated types, you can use `override def checkInputDataTypes` to throw something like `UNSUPPORTED_DATATYPE` if child.dataType is a complex type with **collated** strings @uros-db how about instead, I add support for complex types in this PR and in the next one, try to switch implementation to the mode.update approach. For the pr, i will add a benchmark that calls update as well as eval because the current benchmark won't be relevant to the mode.update approach as it is a benchmark of calls to eval -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
