> Is there anything that obviously stands out as severely unbalanced? The R720XD comes with a H710 - instead of putting them in RAID0, I'm thinking a different HBA might be a better idea, any recommendations please? > Don't know that HBA. Does it support pass through mode or HBA mode?
H710 card does not support pass-through. With a R720 I would recommend a JBOD card for example LSI 9207-8i. With Dell next generation servers (H730XD) they carry H730 wich already have pass-through. El mié., 20 jun. 2018 a las 15:00, Luis Periquito (<periqu...@gmail.com>) escribió: > adding back in the list :) > > ---------- Forwarded message --------- > From: Luis Periquito <periqu...@gmail.com> > Date: Wed, Jun 20, 2018 at 1:54 PM > Subject: Re: [ceph-users] Planning all flash cluster > To: <nick.b...@gmail.com> > > > On Wed, Jun 20, 2018 at 1:35 PM Nick A <nick.b...@gmail.com> wrote: > > > > Thank you, I was under the impression that 4GB RAM per 1TB was quite > generous, or is that not the case with all flash clusters? What's the > recommended RAM per OSD currently? Happy to throw more at it for a > performance boost. The important thing is that I'd like all nodes to be > absolutely identical. > I'm doing 8G per OSD, though I use 1.9T SSDs. > > > > > Based on replies so far, it looks like 5 nodes might be a better idea, > maybe each with 14 OSD's (960GB SSD's)? Plenty of 16 slot 2U chassis around > to make it a no brainer if that's what you'd recommend! > I tend to add more nodes: 1U with 4-8 SSDs per chassis to start with, > and using a single CPU with high frequency. For IOPS/latency cpu > frequency is really important. > I have started a cluster that only has 2 SSDs (which I share with the > OS) for data, but has 8 nodes. Those servers can take up to 10 drives. > > I'm using the Fujitsu RX1330, believe Dell would be the R330, with a > Intel E3-1230v6 cpu and 64G of ram, dual 10G and PSAS (passthrough > controller). > > > > > The H710 doesn't do JBOD or passthrough, hence looking for an > alternative HBA. It would be nice to do the boot drives as hardware RAID 1 > though, so a card that can do both at the same time (like the H730 found > R630's etc) would be ideal. > > > > Regards, > > Nick > > > > On 20 June 2018 at 13:18, Luis Periquito <periqu...@gmail.com> wrote: > >> > >> Adding more nodes from the beginning would probably be a good idea. > >> > >> On Wed, Jun 20, 2018 at 12:58 PM Nick A <nick.b...@gmail.com> wrote: > >> > > >> > Hello Everyone, > >> > > >> > We're planning a small cluster on a budget, and I'd like to request > any feedback or tips. > >> > > >> > 3x Dell R720XD with: > >> > 2x Xeon E5-2680v2 or very similar > >> The CPUs look good and sufficiently fast for IOPS. > >> > >> > 96GB RAM > >> 4GB per OSD looks a bit on the short side. Probably 192G would help. > >> > >> > 2x Samsung SM863 240GB boot/OS drives > >> > 4x Samsung SM863 960GB OSD drives > >> > Dual 40/56Gbit Infiniband using IPoIB. > >> > > >> > 3 replica, MON on OSD nodes, RBD only (no object or CephFS). > >> > > >> > We'll probably add another 2 OSD drives per month per node until full > (24 SSD's per node), at which point, more nodes. We've got a few SM863's in > production on other system and are seriously impressed with them, so would > like to use them for Ceph too. > >> > > >> > We're hoping this is going to provide a decent amount of IOPS, 20k > would be ideal. I'd like to avoid NVMe Journals unless it's going to make a > truly massive difference. Same with carving up the SSD's, would rather not, > and just keep it as simple as possible. > >> I agree: those SSDs shouldn't really require a journal device. Not > >> sure about the 20k IOPS specially without any further information. > >> Doing 20k IOPS at 1kB block is totally different at 1MB block... > >> > > >> > Is there anything that obviously stands out as severely unbalanced? > The R720XD comes with a H710 - instead of putting them in RAID0, I'm > thinking a different HBA might be a better idea, any recommendations please? > >> Don't know that HBA. Does it support pass through mode or HBA mode? > >> > > >> > Regards, > >> > Nick > >> > _______________________________________________ > >> > ceph-users mailing list > >> > ceph-users@lists.ceph.com > >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Félix Barbeira.
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
