Dandandan commented on issue #20078: URL: https://github.com/apache/datafusion/issues/20078#issuecomment-3834368617
Looking a bit more at the profile and `PhysicalExprSimplifier`, I see some major inefficiencies in `simplify_not_expr` / `simplify_unwrap_cast` `simplify_const_expr` I think we can solve before implementing the cache, I think this should reduce it a great bit: * Double recursion in `simplify_unwrap_cast` as the outer does `transform_data` and the inner does `transform_down` leading to O(n^2) runtime - can be avoided by not doing `transform_down` * Double recursion in the `simplify_const_expr` optimizer leading to O(n^2) runtime - can be simplified by only checking things at the root * The 3 rules `Arc::clone` the input when input is unchanged - this can be avoided by passing an `Arc` instead of `&Arc` and removing the clone. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
