Hi David, Could you elaborate on which part of my proposal overlaps with Substrait? I don’t see anything in Substrait that allows me to do something along the lines of
Expression e = Expression::FromString(“(add !.a $int32:1)”); in the code. Sasha > On Oct 5, 2022, at 1:35 PM, Lee, David <david....@blackrock.com.INVALID> > wrote: > > I believe this is what substrait.io <http://substrait.io/> is trying to > accomplish.. > > Here's some additional info: > https://substrait.io/ <https://substrait.io/> > > https://www.youtube.com/watch?v=5JjaB7p3Sjk > <https://www.youtube.com/watch?v=5JjaB7p3Sjk> > > -----Original Message----- > From: Sasha Krassovsky <krassovskysa...@gmail.com > <mailto:krassovskysa...@gmail.com>> > Sent: Wednesday, October 5, 2022 11:29 AM > To: dev@arrow.apache.org <mailto:dev@arrow.apache.org> > Subject: Parser for expressions > > External Email: Use caution with links and attachments > > > Hi everyone, > I’ve noticed on the mailing list a few times people asking for a more > convenient way to construct an Expression, namely using a string of some > sort. I’ve found myself wishing for something like this too when constructing > ExecPlans, and so I’ve gone ahead and implemented a parser [0]. I was > wondering if anyone had any thoughts about the design of the language? > > The current implementation parses a lisp-like language. This language has > three types of expressions (mirroring the current Expression API): > > - A call is a normal s-expression, it has the name of the kernel and the list > of arguments. Its arguments can be any expression. > - A literal (i.e. scalar) starts with a $ and specifies a type and a value, > separated by a colon. For example, `$decimal(12,2):10.01` specifies a literal > of type decimal(12, 2) and a value of 10.01. > - A field_ref starts with a ! and is an identifier in the schema following > the DotPath syntax we already have [1]. > > So for example, the expression > > (add $int32:1 (multiply !.a !.b)) > > computes a*b+1 given a batch with columns named a and b. > > The reason I chose a lisp-like language is that it very directly translates > to the current Expression API and that it feels more natural to use a prefix > notation for a language where all functions have a name (i.e. no +, -, *, > etc.). > > I’m currently working on a followup PR for specifying ExecPlans from a string > (mainly for easier testing), and would like that language to be an extension > of this one. Looking forward to hearing everyone’s thoughts! > > Thanks, > Sasha Krassovsky > > [0] > https://urldefense.com/v3/__https://github.com/apache/arrow/pull/14287__;!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG_6oZdDk$ > > <https://urldefense.com/v3/__https://github.com/apache/arrow/pull/14287__;!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG_6oZdDk$> > > <https://urldefense.com/v3/__https://github.com/apache/arrow/pull/14287__;!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG_6oZdDk$ > > <https://urldefense.com/v3/__https://github.com/apache/arrow/pull/14287__;!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG_6oZdDk$> > > > [1] > https://urldefense.com/v3/__https://github.com/apache/arrow/blob/master/cpp/src/arrow/type.h*L1726__;Iw!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG0GkL0Mn$ > > <https://urldefense.com/v3/__https://github.com/apache/arrow/blob/master/cpp/src/arrow/type.h*L1726__;Iw!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG0GkL0Mn$> > > <https://urldefense.com/v3/__https://github.com/apache/arrow/blob/master/cpp/src/arrow/type.h*L1726__;Iw!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG0GkL0Mn$ > > <https://urldefense.com/v3/__https://github.com/apache/arrow/blob/master/cpp/src/arrow/type.h*L1726__;Iw!!KSjYCgUGsB4!enYRTooMrwyJKJzgTlQMdMhpfT7ys3Ol8a8HcHUvxRYRN-a-Up_axLfPGOpUtEDCDs0ee7lHPAzVdz-dooULG0GkL0Mn$> > > > > > > This message may contain information that is confidential or privileged. If > you are not the intended recipient, please advise the sender immediately and > delete this message. See > http://www.blackrock.com/corporate/compliance/email-disclaimers > <http://www.blackrock.com/corporate/compliance/email-disclaimers> for further > information. Please refer to > http://www.blackrock.com/corporate/compliance/privacy-policy > <http://www.blackrock.com/corporate/compliance/privacy-policy> for more > information about BlackRock’s Privacy Policy. > > > For a list of BlackRock's office addresses worldwide, see > http://www.blackrock.com/corporate/about-us/contacts-locations > <http://www.blackrock.com/corporate/about-us/contacts-locations>. > > © 2022 BlackRock, Inc. All rights reserved.
