tustvold commented on PR #445: URL: https://github.com/apache/arrow-rs-object-store/pull/445#issuecomment-3133534628
object_store will retry PUT requests provided they don't have preconditions - https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/mod.rs#L182. My suspicion is slatedb is specifying a precondition and therefore the retry machinery doesn't retry - https://github.com/apache/arrow-rs-object-store/blob/main/src/client/retry.rs#L439. _Whilst I am aware that technically a put with preconditions is idempotent as idempotency refers to the server-side state not the client-side result, in practice few clients would handle it correctly if we retried such requests._ > Perhaps they shouldn't and just implement their own retry policies? Slatedb should likely not be solely relying on object_store retries, there will almost certainly be error scenarios where recovery would require retrying at a higher level. > elapsed: 30.000417786s As an aside, this would suggest that slatedb is using PUT for very large payloads where it should possibly be using a multipart upload instead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
