On Wed, Oct 11, 2017 at 3:46 PM, Jialin Liu <jaln...@lbl.gov> wrote: > Hi, > I'm new to openstack swift, I'm a HPC user, by several days of exploration > of swift, I have some naive questions: > 1. Can swift, e.g., PUT, leverage OS' page buffer?; >
Sure, but perhaps to a limited degree depending on what you're expecting? We try pretty hard to fsync everything before we return success: https://github.com/openstack/swift/blob/master/swift/obj/diskfile.py#L1517 ^ and just below that is _finalize_put > Is there a Linux kernel module for swift? > No. > What are the existing optimizations for caching the write in order to gain > better bandwidth on spin disks? > ... um ... that's a pretty broad question, swift has been around for quite a while, it would require some research. You can read through the diskfile modules; most of the bits that touch disk at the object layer are in there. There's a number of tunables that effect how Swift will treat the filesystem - and the low-level specifics of what they do are in the code. Any results of rigorous classifications you perform and want to publish are always interesting and welcome in the community. There's been a few interesting things published over the years - but there's no been no community effort to collect them into a single home that I'm aware of; you'd have to track them down - I recall Seagate did some interesting analysis a while back. RedHat's performance group is always doing stuff, Intel did some stuff. Current community efforts are focused on further refining erasure code storage - which have a positive impact on medium and large object uploads both by reducing the total number of backend bytes to store (compared to replicated object) and also by fanning that object data + parity out to larger numbers of spindles on the backend. On the other end of the spectrum OVH is leading an effort to further improve performance for lots of small files. 2. Does swift support asynchronous PUT/Get? > > I don't know what that means, so I'll say no. I might hazard a guess it has something to do with a PUT only storing some reduced redundancy unsafe staged data and then doing something to improve durability after it already promised the client their write was "safe" - which is not something Swift does. > > Also please let me know if the dev list is good for me to ask this kind of > questions. > The ML is nice in that people can give more detailed responses and the archives tend to be a bit more searchable - the trade off is a longer latency on responses. You can also jump on IRC - swift folks hang out in #openstack-swift on freenode. Best Regards, Clay
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
