I started down that path and got so deep that I couldn't even find where I went in. I couldn't make heads or tails out of what would or wouldn't work.
We didn't need multiple hosts accessing a single datastore, so on the client side I just have a single VM guest running on each ESXi hosts, with the cephfs filesystem mounted on it(via a 10Gb connection to the ceph environment), and then exported via NFS on a host-only network, and mounted on the host. Not quite as redundant as it could be, but good enough for our usage. I'm seeing ~500MB/s speeds going to a 4-node cluster with 5x1TB 7200rpm drives. I tried it first, in a similar config, except using LIO to export an RBD device via iSCSI, still on the local host network. Write performance was good, but read performance was only around 120MB/s. I didn't do much troubleshooting, just tried NFS after that and was happy with it. On Tue, Jun 6, 2017 at 2:33 AM, Adrian Saul <adrian.s...@tpgtelecom.com.au> wrote: > > > Early usage will be CephFS, exported via NFS and mounted on ESXi 5.5 > > > and > > > 6.0 hosts(migrating from a VMWare environment), later to transition to > > > qemu/kvm/libvirt using native RBD mapping. I tested iscsi using lio > > > and saw much worse performance with the first cluster, so it seems > > > this may be the better way, but I'm open to other suggestions. > > > > > I've never seen any ultimate solution to providing HA iSCSI on top of > Ceph, > > though other people here have made significant efforts. > > In our tests our best results were with SCST - also because it provided > proper ALUA support at the time. I ended up developing my own pacemaker > cluster resources to manage the SCST orchestration and ALUA failover. In > our model we have a pacemaker cluster in front being an RBD client > presenting LUNs/NFS out to VMware (NFS), Solaris and Hyper-V (iSCSI). We > are using CephFS over NFS but performance has been poor, even using it just > for VMware templates. We are on an earlier version of Jewel so its > possibly some later versions may improve CephFS for that but I have not had > time to test it. > > We have been running a small production/POC for over 18 months on that > setup, and gone live into a much larger setup in the last 6 months based on > that model. It's not without its issues, but most of that is a lack of > test resources to be able to shake out some of the client compatibility and > failover shortfalls we have. > > Confidentiality: This email and any attachments are confidential and may > be subject to copyright, legal or some other professional privilege. They > are intended solely for the attention and use of the named addressee(s). > They may only be copied, distributed or disclosed with the consent of the > copyright owner. If you have received this email by mistake or by breach of > the confidentiality clause, please notify the sender immediately by return > email and delete or destroy all copies of the email. Any confidentiality, > privilege or copyright is not waived or lost because this email has been > sent to you by mistake. > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
