skellys opened a new pull request, #115: URL: https://github.com/apache/iceberg-python/pull/115
I found a thread safety bug where some unwanted partitions would make it past the partition filter. I found and tested this fix on a simple Iceberg table with one schema spec, 5 manifest files (so 5 threads used in the thread pool), and thousands of partitions. I've also solved this with a thread local storage approach to create fewer _ExpressionEvaluator instances under the hood, but I didn't think that'd be worth the additional complexity, especially with per-thread cleanup. TODO - [ ] discuss approach on how to test this in the test suite itself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
