dannypage commented on issue #1335: URL: https://github.com/apache/iceberg-python/issues/1335#issuecomment-2936111497
Hi folks! We are loving Iceberg and PyIceberg makes it a lot more accessible. We are currently doing a massive backfill (50k files per table) and seeing ~100-500 files per minute in S3 when working with batches of 1000 files. Going to test with the nightly build to see if there will be a performance, but I was curious about two things: - Will this Issue be included in the `0.10` release? - Is there a science to the ideal batch size for `add_files`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
