Yep, you're right. Doing a HEAD request before every PUT gets expensive, especially for small files.
But don't despair! There's some good news. First, realize that swiftclient is written for a pretty general use case. If you have more knowledge about how your system works, then you can write something more efficient. python-swiftclient has a CLI, yes, but it also provides a python module so that you can write your own client as needed. For example, you can write your own script that doesn't do HEAD requests before a PUT. Second, this specific issue is being addressed in patch https://review.openstack.org/#/c/81646/ which adds If-None-Match support to PUT requests in Swift. This will allow a PUT without a HEAD and also avoid data transfer if the object already exists. I hope this patch lands today and is included in Icehouse. --John On Apr 2, 2014, at 7:52 AM, Roman Kravets <soft...@gmail.com> wrote: > Hello. > > I have trouble with python-swiftclient during upload to match small > file to Openstack Swift. > > It client make HEAD request for each file during upload before PUT. It > make very intencive read load to hard drive and as result all cluster > show very small upload speed. > > I see that for each file it make 6 HEAD request (3 recplicas * 2). > > When I use cloudfuse this client make only PUT and as result hard > drive load only to write data. It is make less IOps to hard drive. > > What need make HEAD request for each files? How can I reduce the read > request for reduce load to hard drive? > > Thank you! > > -- > Best regards, > Roman Kravets > > _______________________________________________ > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > Post to : openstack@lists.openstack.org > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
