Hi Jeff, Currently, there is no open source backend in manila that provides scalable and highly-available NFS servers for dynamic cloud workloads. Manila's CephFS driver could integrate with your on-going work on active-active NFS over CephFS (with Kubernetes managing the lifecycle of containerized user-space NFS-Ganesha servers) [1] to fill this gap.
During the manila project team gathering, we discussed this plan under the topic of high availability of share servers [2]. One of the questions was about write integrity issue when a NFS-Ganesha server container goes down and another container comes up to replace it. Would there be any such write integrity issues when NFS clients do asynchronous writes to files with write caching in the NFS client and the NFS server (NFS-Ganesha server side caching or the libcephfs client caching), and the NFS-Ganesha server goes down? I guess this a general NFS protocol question or maybe things get complicated with NFS-Ganesha over CephFS? I looked up the NFSv4 protocol documentation, the implementation of COMMIT operation [3]. So if a NFS client issues a aysnc write followed by a COMMIT operation that succeeds, then it's expected that the NFS server has flushed cached data and metadata onto stable storage, here CephFS. And if the NFS server crashes losing cached data and metadata, then the write verifier cookie returned by the WRITE or COMMIT operation indicates to the client that the server crashed. Now it's up to the NFS client to re-transmit the uncached data and metadata. Thanks, Ramana [1] https://jtlayton.wordpress.com/2017/11/07/active-active-nfs-over-cephfs/ [2] line 186 in https://etherpad.openstack.org/p/manila-rocky-ptg the actual spec https://review.openstack.org/#/c/504987/ [3] https://tools.ietf.org/html/rfc7530#section-16.3.5 __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev