[ https://issues.apache.org/jira/browse/ARROW-17820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629225#comment-17629225 ]
Weston Pace edited comment on ARROW-17820 at 11/4/22 11:24 PM: --------------------------------------------------------------- {quote} Such an approach doesn't really fit our kernels / Acero, I think. One option could be to have a generic kernel to "map" another kernel on the list values. where you can pass the function name you want to apply, and a FunctionOptions object matching the kernel. Would something like this be possible technically? {quote} Yes, I think that should be possible for unary kernels. Though I think mapping a single kernel (as opposed to a single expression) might be a bit limiting, though maybe it isn't so bad. For example, what if a user wants to do something like {{map(lambda f: f.upper() * 2, ["a", "b", "c"])}} Another thing is that it should be valid to use n-ary functions too provided the other arguments are scalars. This discussion has come up in Substrait with respect to lambdas (https://github.com/substrait-io/substrait/issues/349). Perhaps the "map function" for {{List<T>}} could be an expression bound to a schema of "\{item: T\}" (e.g. so you could do {{field_ref(0)}} or {{field_ref("item")}}). Though if the map function is an expression then a kernel would have to execute an entire expression which may or may not be doable (I've reached the limit of my imagination for a Friday :) was (Author: westonpace): {quote} Such an approach doesn't really fit our kernels / Acero, I think. One option could be to have a generic kernel to "map" another kernel on the list values. where you can pass the function name you want to apply, and a FunctionOptions object matching the kernel. Would something like this be possible technically? {quote} Yes, I think that should be possible for unary kernels. Though I think mapping a single kernel (as opposed to a single expression) might be a bit limiting, though maybe it isn't so bad. For example, what if a user wants to do something like {{map(lambda f: f.upper() * 2, ["a", "b", "c"])}} Another thing is that it should be valid to use n-ary functions too provided the other arguments are scalars. This discussion has come up in Substrait with respect to lambdas (https://github.com/substrait-io/substrait/issues/349). Perhaps the "map function" for {{List<T>}} could be an expression bound to a schema of "{item: T}" (e.g. so you could do {{field_ref(0)}} or {{field_ref("item")}}). Though if the map function is an expression then a kernel would have to execute an entire expression which may or may not be doable (I've reached the limit of my imagination for a Friday :) > [C++] Implement arithmetic kernels on List(number) > -------------------------------------------------- > > Key: ARROW-17820 > URL: https://issues.apache.org/jira/browse/ARROW-17820 > Project: Apache Arrow > Issue Type: New Feature > Components: C++, Python > Reporter: Adam Lippai > Priority: Major > Labels: kernel, query-engine > > eg. rounding in list(float64()), similar to a map or foreach -- This message was sent by Atlassian Jira (v8.20.10#820010)