I'm +1 for anything that adds support for subexpression elimination to conditional expressions, I've been trying to get that support in for a long time https://github.com/apache/spark/pull/32987. This is a major performance limitation in Spark in my opinion.
On Tue, Mar 7, 2023 at 4:24 PM Kun Wan <wanku...@163.com> wrote: > Hi, Mich > Thanks for your reminder. > It seems "Support for subexpression elimination with more physical > operators" should not be a SPIP, it's just a design doc. > I have changed the title of the document. > Hope someone who are interested will join in and complete it together. > > At 2023-03-08 01:50:17, "Mich Talebzadeh" <mich.talebza...@gmail.com> > wrote: > > > Hi Kun, > > > Have you checked the doc procedure for SPIP here in case > > > Spark Project Improvement Proposals (SPIP) | Apache Spark > <https://spark.apache.org/improvement-proposals.html> > > > HT > > > view my Linkedin profile > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > https://en.everybodywiki.com/Mich_Talebzadeh > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Tue, 7 Mar 2023 at 13:32, 万昆 <wanku...@163.com> wrote: > >> Hi everyone, >> >> >> I would like to start a discussion on "Subexpression elimination >> supporting more physical operators" >> >> >> Now spark only support subexpression elimination in ProjectExec and >> AggregateExec. We can support improve subexpression elimination framework >> to support more physical operators. >> >> >> SPIP Jira: https://issues.apache.org/jira/browse/SPARK-42551 >> >> SPIP Doc: >> https://docs.google.com/document/d/165cv7hRvkFvuUHlnbapWvamxcxcn9FJj/edit?usp=sharing&ouid=107277827304520252190&rtpof=true&sd=true >> >> >> Thank you >> >> -- Adam Binford
