alamb opened a new pull request, #7650:
URL: https://github.com/apache/arrow-rs/pull/7650

   # Which issue does this PR close?
   
   We generally require a GitHub issue to be filed for all bug fixes and 
enhancements and this helps us generate change logs for our releases. You can 
link an issue to this PR using the GitHub syntax.
   
   Closes #NNN.
   
   # Rationale for this change
   
   Currently coalesce buffers views / data until there are enough rows and then 
concat's the results together
   
   This is wasteful because it
   1. Buffers memory (has 2x the peak usage)
   2. Copies the data twice
   
   We can make it faster and more memory efficient by directly creating the 
output array
   
   # What changes are included in this PR?
   
   Note this PR does NOT yet add specialized filtering -- instead it focuses on 
reducing the
   overhead of appending views. 
   
   # Open questions:
   1. There is substantial overlap / duplication with StringViewBuilder
   
   What is different:
   1. Block size calculation management (aka look at the buffer sizes of the 
incoming buffers)
   2. Finishing array allocates sufficient space for views
   
   # Are there any user-facing changes?
   
   If there are user-facing changes then we may require documentation to be 
updated before approving the PR.
   
   If there are any breaking changes to public APIs, please call them out.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to