kevinjqliu commented on issue #2325: URL: https://github.com/apache/iceberg-python/issues/2325#issuecomment-3797145825
Hey all, thanks for all the detailed discussion here. Summarizing from what I've read above - the manifest cache is problematic; it's utilizing a lot of memory, esp in the case for fast append - theres a potential memory leak in Avro reader I've opened #2951 to cache the ManifestFile We should fix the avro reader leak as well Its probably a good idea to use the script above and create a benchmark script to test for memory leaks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
