Ok, so ipoib is required... ><> nathan stratton
On Mon, Jan 6, 2020 at 4:45 AM Wei Zhao <zhao6...@gmail.com> wrote: > From my understanding, the basic idea is that ceph exchange rdma > information(qp,gid and so) through ip address on rdma device, and then > communicate with each other throng rdma. But in my tests, there > seemed to be some issues in that codes. > > On Fri, Jan 3, 2020 at 2:24 AM Nathan Stratton <nat...@robotics.net> > wrote: > > > > I am working on upgrading my current ethernet only ceph cluster to a > combined ethernet frontend and infiniband backend. From my research I > understand that I set: > > > > ms_cluster_type = async+rdma > > ms_async_rdma_device_name = mlx4_0 > > > > What I don't understand is how does ceph know how to reach each OSD over > RDMA? Do I have to run IPoIB on top of infiniband and use that for OSD > addresses? > > > > Is there a way to use infiniband on backend without IPoIB and just use > rdma verbs? > > > > ><> > > nathan stratton > > _______________________________________________ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
