zhuqi-lucas commented on issue #6692: URL: https://github.com/apache/arrow-rs/issues/6692#issuecomment-2900627714
> I have been playing around with various options for this feature in the context of [#7456](https://github.com/apache/arrow-rs/issues/7456) where I want to save the results of applying a filter with minimal buffering > > * [POC: Sketch out cached filter result API #7513](https://github.com/apache/arrow-rs/pull/7513) > > ## We already have the `concat` method on some builders > Several of the array builders already have a [`append_array`](https://docs.rs/arrow/latest/arrow/index.html?search=append_array) function > >  > > Adding some new `concat_array` methods for other builders seems reasonable and follows the existing patterns in this crate > > ## Could expose the contents of `take` and `filter` as extension traits on `Builders` > I was thinking we could potentially add new `filter` methods on the builders using the same implementations as the filter kernel > > pub trait BuilderFilterExt { > /// Appends all values from `array` to the current builder where `filter` is Some(true) > /// Rows where `filter` is SOme(false) or None will not be copied > pub fn append_filter(&mut self, array: &ArrayRef, filter: &BooleanArray) > } > I think we could likely the refactor the filter kernels to potentially use that trait internally. > > The same thing can be done for `take` > > I plan to try prototyping it to see if it makes a difference in the context of Great idea @alamb , looking forward the prototype! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
