amoeba commented on issue #38032: URL: https://github.com/apache/arrow/issues/38032#issuecomment-1773486968
Hi @kostovasandra, a few thoughts: - Can you elaborate on how RDS is involved here? Looking at your code, I don't see any functions related to reading/writing RDS files, though I do see that your code uses the string 'rds' in a few places, despite using only `read_parquet` and `write_parquet` - Have you tried reading the file on other S3-compatible storage and possibly even from your local machine? - Without being able to turn on S3 logging at the moment, could you find some way to track how many packets and how much data is transferred during the 10 minutes? - Can you try any other files on the same storage or a subset of your whole file? I wonder if all reads are slow or if there's some pattern -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
