Fokko commented on issue #6475: URL: https://github.com/apache/iceberg/issues/6475#issuecomment-1404692576
Looking at the recent improvements by @rdblue Before 887 requests, after the change 203 requests, which is a great improvement, but still more to be done! After: ``` ➜ ~ time python3 /tmp/vo.py python3 /tmp/vo.py 2.10s user 2.02s system 77% cpu 5.294 total python3 /tmp/vo.py 2.56s user 1.27s system 68% cpu 5.558 total python3 /tmp/vo.py 1.96s user 1.94s system 68% cpu 5.729 total ``` Before: ``` python3 /tmp/vo.py 2.15s user 1.86s system 64% cpu 6.232 total python3 /tmp/vo.py 2.61s user 1.53s system 65% cpu 6.315 total python3 /tmp/vo.py 2.48s user 1.58s system 66% cpu 6.081 total ``` Keep in mind that I'm testing against a local minio, so when using high-latency object storage such as s3 will make it much worse. Before: https://gist.github.com/Fokko/495dce8234148be74eb312d1af93e694 After: https://gist.github.com/Fokko/a0cc4ac4ec6a7701a145a4a3da3a6bc6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
