Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-17 Thread via GitHub
crepererum commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2117094510 I can see if I get organize you some profiles next week :slightly_smiling_face: -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-16 Thread via GitHub
erratic-pattern commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2116144023 > I think there's nothing wrong about having a "display" infrastructure, but it shouldn't be used eagerly. As others pointed out, using a hash or any form of numeric ID

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-15 Thread via GitHub
alamb commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2113109684 > FWIW I've also seen the high cost of expression string formatting (using `Display`/`to_string()`) in a good number of profiles. > I think there's nothing wrong about havin

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-15 Thread via GitHub
crepererum commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2112020903 FWIW I've also seen the high cost of expression string formatting (using `Display`/`to_string()`) in a good number of profiles. I think there's nothing wrong about havi

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-14 Thread via GitHub
peter-toth commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2110285907 > Here is one example API that I would love to implement with such a tree-node api: https://github.com/apache/datafusion/issues/10505 Thanks for sharing this @alamb. It

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-14 Thread via GitHub
alamb commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2110217302 > I have a WIP commit here: https://github.com/peter-toth/arrow-datafusion/commit/e8447996462ae4710b573c7088bc6d8b1e586cfb that adds TreeNode::apply_ref() / TreeNode::visit_ref().

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-12 Thread via GitHub
peter-toth commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2106334918 I've opened a draft PR: https://github.com/apache/datafusion/pull/10473 and will try to wrap it up in the following days. -- This is an automated message from the Apache G

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-11 Thread via GitHub
peter-toth commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2105745741 > I like the idea of generalizing the `(u64, &Expr)` struct into something reuseable across optimizations. Honestly, I don't know the those referenced usecases, but I f

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-11 Thread via GitHub
erratic-pattern commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2105737013 Thanks for the detailed write up @peter-toth . Though I did mention `HashSet` specifically, my suggestion more generally goes along the lines of using the `Hash` implem

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-11 Thread via GitHub
peter-toth commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2105664520 > Are there any potential issues with simply using the existing `Hash` implementation of `Expr` to create `HashSet`s? > > Serveral other optimization passes use string

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-10 Thread via GitHub
erratic-pattern commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2105361124 Are there any potential issues with simply using the existing `Hash` implementation of `Expr` to create `HashSet`s? Serveral other optimization passes use string n

Re: [I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-08 Thread via GitHub
peter-toth commented on issue #10426: URL: https://github.com/apache/datafusion/issues/10426#issuecomment-2101062450 I'm happy to take this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[I] Make `CommonSubexprEliminate` faster by avoiding the use of strings [datafusion]

2024-05-08 Thread via GitHub
alamb opened a new issue, #10426: URL: https://github.com/apache/datafusion/issues/10426 ### Is your feature request related to a problem or challenge? Part of https://github.com/apache/datafusion/issues/5637 One of the optimizer passes is "common subexpression elimination" that