alamb opened a new issue, #8897:
URL: https://github.com/apache/arrow-rs/issues/8897

   **Describe the bug**
   
   While testing the upgrade to arrow 57.1.0 in DataFusion
   - https://github.com/apache/datafusion/pull/18820/files
   
   I noticed that the reported size of the memory used by metadata grew by 
almost 33% 
   
   <img width="2099" height="320" alt="Image" 
src="https://github.com/user-attachments/assets/668bf911-cb2b-4955-9118-82954ba65278";
 />
   
   I know that @adamreeve fixed the heap accounting for encryption 
   - https://github.com/apache/arrow-rs/pull/8671
   
   But I don't think that would increase usage by 33%
   
   I went back and reviewed the code and I found at least one issue where the 
heap size is double counted for `Arc<T>` I think
   
   
   **Expected behavior**
   Heap usage should be accurate
   
   **Additional context**
   <!--
   Add any other context about the problem here.
   -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to