senzzzi commented on PR #817: URL: https://github.com/apache/arrow-rs-object-store/pull/817#issuecomment-5320662112
> You can also use Mulit-part upload to achieve something similar (though most object stores would still require you to buffer the parts in 5mb chunks) > > We haven't released these docs yet, btu here is an example of how to do it > > <img alt="Screenshot 2026-08-17 at 5 43 00 PM" width="1095" height="1341" src="https://private-user-images.githubusercontent.com/490673/637174766-cbecaa5a-638f-40f4-9f19-c0359764fc89.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODcwMDM0ODEsIm5iZiI6MTc4NzAwMzE4MSwicGF0aCI6Ii80OTA2NzMvNjM3MTc0NzY2LWNiZWNhYTVhLTYzOGYtNDBmNC05ZjE5LWMwMzU5NzY0ZmM4OS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjYwODE3JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI2MDgxN1QyMTQ2MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iNTkxYThiMTVmMjBkMzlhOTZiMzg4MGUxMDRkNjI1YjMxZTgwZTllZjljMzc0NDg2N2JiOTBiMjhhMTU1NzFhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZyZXNwb25zZS1jb250ZW50LXR5cGU9aW1hZ2UlMkZwbmcifQ.4t0l8Zynrr0ZjK_hvCD-Nuosxx3yJDQKuNM4t-Pvcks";> It's not the same because of costs. Uploading a 64Mb file in a single put request is still quite fast (depending on your needs) and you are charged for 1 request. If you upload 64 different 1Mb parts, you are charged by the initial request, 64 parts individually and 1 final requests. So it's 66 times more expensive. You can check the link in this comment: https://github.com/apache/arrow-rs-object-store/issues/281#issuecomment-5123463473 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
![https://private-user-images.githubusercontent.com/490673/637174766-cbecaa5a-638f-40f4-9f19-c0359764fc89.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODcwMDM0ODEsIm5iZiI6MTc4NzAwMzE4MSwicGF0aCI6Ii80OTA2NzMvNjM3MTc0NzY2LWNiZWNhYTVhLTYzOGYtNDBmNC05ZjE5LWMwMzU5NzY0ZmM4OS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjYwODE3JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI2MDgxN1QyMTQ2MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iNTkxYThiMTVmMjBkMzlhOTZiMzg4MGUxMDRkNjI1YjMxZTgwZTllZjljMzc0NDg2N2JiOTBiMjhhMTU1NzFhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZyZXNwb25zZS1jb250ZW50LXR5cGU9aW1hZ2UlMkZwbmcifQ.4t0l8Zynrr0ZjK_hvCD-Nuosxx3yJDQKuNM4t-Pvcks"](https://private-user-images.githubusercontent.com/490673/637174766-cbecaa5a-638f-40f4-9f19-c0359764fc89.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODcwMDM0ODEsIm5iZiI6MTc4NzAwMzE4MSwicGF0aCI6Ii80OTA2NzMvNjM3MTc0NzY2LWNiZWNhYTVhLTYzOGYtNDBmNC05ZjE5LWMwMzU5NzY0ZmM4OS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjYwODE3JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI2MDgxN1QyMTQ2MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iNTkxYThiMTVmMjBkMzlhOTZiMzg4MGUxMDRkNjI1YjMxZTgwZTllZjljMzc0NDg2N2JiOTBiMjhhMTU1NzFhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZyZXNwb25zZS1jb250ZW50LXR5cGU9aW1hZ2UlMkZwbmcifQ.4t0l8Zynrr0ZjK_hvCD-Nuosxx3yJDQKuNM4t-Pvcks"){kind=link}