acvictor commented on PR #5632: URL: https://github.com/apache/incubator-gluten/pull/5632#issuecomment-2152985875
> Can you comment a bit how these info are used by Velox? Is it aligned with Spark's behavior? In Spark, properties are present in the split and obtained during listing. Earlier Velox would make one additional call to remote storage to fetch file length per openFileForRead call on a path even though this information is already present upstream wihle constructing the split. Velox now allows these values to be passed from the caller. By making this change we can eliminate one additional call to remote storage per path (RTT is in the order of 10s of milli seconds). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
