mapleFU commented on issue #38032: URL: https://github.com/apache/arrow/issues/38032#issuecomment-1755560358
Hmm sorry for being confusion. This is a bit confusing because I don't know how many requests have send to s3, and how they looks like. 10s is an optimization, I think it's because `Default` prefetch driver works a bit better than Lazy, since it will try it best to send io requests. But I don't know how these requests takes. I'm not so familiar with s3 SDK, so might take time to debugging, so sorry for that. would you mind enable TRACE as log level ( https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/logging.html and set `initialize_s3(pyarrow._s3fs.S3LogLevel.Trace)` ) ? I just want to know the io-merging status. Sorry again for low efficient -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
