bentsku commented on issue #6799: URL: https://github.com/apache/arrow-rs/issues/6799#issuecomment-2502344533
I've started working on the LocalStack implementation of S3 PutObject and CompleteMultipartUpload `IfMatch`, I have everything working now, just a matter of updating our internal specs, so it might be ready tomorrow evening. I just wanted to share a finding I had around the `CompleteMultipartUpload` behavior that might be unexpected, I haven't looked at your implementation and you might already cover this. Similarly to `IfNoneMatch` behavior with deletion of the object between a `CreateMultipartUpload` and `CompleteMultipartUpload` which still requires a full "restart" of the process as documented on the docs in AWS: > Conditional writes with If-None-Match header > [...] > When using CompleteMultipartUpload, the entire multipart upload must be re-initiated with CreateMultipartUpload to upload the object again after receiving a 409 Conflict error. However, something that is undocumented for `IfMatch` is that it has a similar behavior for `PutObject` requests between `Create` and `Complete`: If you create an object with a `A` key with the fictional ETag `123`, create a multipart upload, upload a new object still with the `A` key with either a different ETag `456` or even with same content and thus `123` ETag, the Complete operation will fail with 409. The object needs to be "untouched" between the Create and Complete part for the operation to succeed. AWS says that 409 exceptions should be retried, but I've tried and this one also necessitates a "restart" of the multipart upload operation. I suspect they are also comparing the "initiated" time of the multipart upload to the current object, and if the current object is newer, they are raising a 409 exception. I'm not too familiar with other cloud providers and I don't know they behavior, and I'm also not too familiar with conditional writes in itself so maybe this behavior is expected and actually wanted, but it surprised me, so I wanted to share 😄 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
