Re: [ceph-users] Intel P3700 SSD for journals
You wrote P3700 so that’s what I discussed ;) If you want to connect to your HBA you’ll want a SATA device like the S3710 series: http://ark.intel.com/products/family/83425/Data-Center-SSDs#@Server The P3700 is a PCI device, goes into an empty slot, and is not speed-limited by the SATA interface. At perhaps higher cost. With 7.2 I would think you’d be fine, driver-wise. Either should be detected and work out of the box. — Anthony > > thx Alan and Anthony for sharing on these P3700 drives. > > Anthony, just to follow up on your email: my OS is CentOS7.2. Can > you please elaborate on nvme on the CentOS7.2, I'm in no way expert on > nvme, but I can here see that > https://www.pcper.com/files/imagecache/article_max_width/news/2015-06-08/Demartek_SFF-8639.png > the connectors are different for nvme. Does this mean I cannot connect > to PERC 730 raid controller? > > Is there anything particular required when installing the CentOS on > these drives, or they will be automatically detected and work out of > the box by default? Thx will > > On Mon, Nov 21, 2016 at 12:16 PM, Anthony D'Atri wrote: >> The SATA S3700 series has been the de-facto for journals for some time. And >> journals don’t neeed all that much space. >> >> We’re using 400GB P3700’s. I’ll say a couple of things: >> >> o Update to the latest firmware available when you get your drives, qual it >> and stick with it for a while so you have a uniform experience >> o Run a recent kernel with a recent nvme.ko, eg. the RHEL 7.1 3.10.0-229.4.2 >> kernel’s bundled nvme.ko has a rare timing issue that causes us resets at >> times. YMMV. >> >> Which OS do you run? >> >> >> >> Read through this document or a newer version thereof >> >> https://www-ssl.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-dc-p3700-spec.pdf >> >> or for SATA drives >> >> http://www.intel.com/content/www/us/en/solid-state-drives/ssd-dc-s3710-spec.html >> >> >> It’s possible that your vendor is uninformed or lying, trying to upsell you. >> At times larger units can perform better due to internal parallelism, ie. a >> 1.6TB unit may electrically be 4x 400GB parts in parallel. For 7200RPM LFF >> drives, as Nick noted 12x journals per P3700 is probably as high as you want >> to go, otherwise you can bottleneck. >> >> What *is* true is the distinction among series. Check the graph halfway >> down this page: >> >> http://www.anandtech.com/show/8104/intel-ssd-dc-p3700-review-the-pcie-ssd-transition-begins-with-nvme >> >> Prima fascia the P3500’s can seem like a relative bargain, but attend to the >> durability — that is where the P3600 and P3700 differ dramatically. For >> some the P3600 may be durable enough, given certain workloads and expected >> years of service. I tend to be paranoid and lobbied for us to err on the >> side of caution with the P3700. YMMV. >> >> — Anthony ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Intel P3700 SSD for journals
thx Alan and Anthony for sharing on these P3700 drives. Anthony, just to follow up on your email: my OS is CentOS7.2. Can you please elaborate on nvme on the CentOS7.2, I'm in no way expert on nvme, but I can here see that https://www.pcper.com/files/imagecache/article_max_width/news/2015-06-08/Demartek_SFF-8639.png the connectors are different for nvme. Does this mean I cannot connect to PERC 730 raid controller? Is there anything particular required when installing the CentOS on these drives, or they will be automatically detected and work out of the box by default? Thx will On Mon, Nov 21, 2016 at 12:16 PM, Anthony D'Atri wrote: > The SATA S3700 series has been the de-facto for journals for some time. And > journals don’t neeed all that much space. > > We’re using 400GB P3700’s. I’ll say a couple of things: > > o Update to the latest firmware available when you get your drives, qual it > and stick with it for a while so you have a uniform experience > o Run a recent kernel with a recent nvme.ko, eg. the RHEL 7.1 3.10.0-229.4.2 > kernel’s bundled nvme.ko has a rare timing issue that causes us resets at > times. YMMV. > > Which OS do you run? > > > > Read through this document or a newer version thereof > > https://www-ssl.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-dc-p3700-spec.pdf > > or for SATA drives > > http://www.intel.com/content/www/us/en/solid-state-drives/ssd-dc-s3710-spec.html > > > It’s possible that your vendor is uninformed or lying, trying to upsell you. > At times larger units can perform better due to internal parallelism, ie. a > 1.6TB unit may electrically be 4x 400GB parts in parallel. For 7200RPM LFF > drives, as Nick noted 12x journals per P3700 is probably as high as you want > to go, otherwise you can bottleneck. > > What *is* true is the distinction among series. Check the graph halfway down > this page: > > http://www.anandtech.com/show/8104/intel-ssd-dc-p3700-review-the-pcie-ssd-transition-begins-with-nvme > > Prima fascia the P3500’s can seem like a relative bargain, but attend to the > durability — that is where the P3600 and P3700 differ dramatically. For some > the P3600 may be durable enough, given certain workloads and expected years > of service. I tend to be paranoid and lobbied for us to err on the side of > caution with the P3700. YMMV. > > — Anthony ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Intel P3700 SSD for journals
The SATA S3700 series has been the de-facto for journals for some time. And journals don’t neeed all that much space. We’re using 400GB P3700’s. I’ll say a couple of things: o Update to the latest firmware available when you get your drives, qual it and stick with it for a while so you have a uniform experience o Run a recent kernel with a recent nvme.ko, eg. the RHEL 7.1 3.10.0-229.4.2 kernel’s bundled nvme.ko has a rare timing issue that causes us resets at times. YMMV. Which OS do you run? Read through this document or a newer version thereof https://www-ssl.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-dc-p3700-spec.pdf or for SATA drives http://www.intel.com/content/www/us/en/solid-state-drives/ssd-dc-s3710-spec.html It’s possible that your vendor is uninformed or lying, trying to upsell you. At times larger units can perform better due to internal parallelism, ie. a 1.6TB unit may electrically be 4x 400GB parts in parallel. For 7200RPM LFF drives, as Nick noted 12x journals per P3700 is probably as high as you want to go, otherwise you can bottleneck. What *is* true is the distinction among series. Check the graph halfway down this page: http://www.anandtech.com/show/8104/intel-ssd-dc-p3700-review-the-pcie-ssd-transition-begins-with-nvme Prima fascia the P3500’s can seem like a relative bargain, but attend to the durability — that is where the P3600 and P3700 differ dramatically. For some the P3600 may be durable enough, given certain workloads and expected years of service. I tend to be paranoid and lobbied for us to err on the side of caution with the P3700. YMMV. — Anthony ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Intel P3700 SSD for journals
We use the 800GB version as journal devices with up to an 1:18 ratio and have had good experiences no bottleneck on the journal side. These also feature good endurance characteristics. I would think that higher capacities are hard to justify as journals -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Heath Albritton Sent: Friday, November 18, 2016 9:19 AM To: ceph-users@lists.ceph.com Subject: Re: [ceph-users] Intel P3700 SSD for journals I've used the 400GB unit extensively for almost 18 months, one per six drives. They've performed flawlessly. In practice, journals will typically be quite small relative to the total capacity of the SSD. As such, there will be plenty of room for wear leveling. If there was some concern, one could over-provision them even more than they already are. -H > On Nov 18, 2016, at 05:43, William Josefsson > wrote: > > Hi list, I wonder if there is anyone who have experience with Intel > P3700 SSD drives as Journals, and can share their experience? > > I was thinking of using the P3700 SSD 400GB as journal in my ceph > deployment. It is benchmarked in Sebastian hann ssd page as well. > However a vendor I spoke to didn't qualify the small sizes of this > model as "enterprise grade/warranty". They suggested the 1.8TB or 2TB. > I have asked for clarification on why this is the case. > > Has anyone experienced any issues with the smaller size P3700 SSDs, > and I'm not sure how a smaller drive could affect the quality of the > product? Maybe anyone can shed light on if the size of the SSD drive > matters? thx will ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Intel P3700 SSD for journals
I've used the 400GB unit extensively for almost 18 months, one per six drives. They've performed flawlessly. In practice, journals will typically be quite small relative to the total capacity of the SSD. As such, there will be plenty of room for wear leveling. If there was some concern, one could over-provision them even more than they already are. -H > On Nov 18, 2016, at 05:43, William Josefsson > wrote: > > Hi list, I wonder if there is anyone who have experience with Intel > P3700 SSD drives as Journals, and can share their experience? > > I was thinking of using the P3700 SSD 400GB as journal in my ceph > deployment. It is benchmarked in Sebastian hann ssd page as well. > However a vendor I spoke to didn't qualify the small sizes of this > model as "enterprise grade/warranty". They suggested the 1.8TB or 2TB. > I have asked for clarification on why this is the case. > > Has anyone experienced any issues with the smaller size P3700 SSDs, > and I'm not sure how a smaller drive could affect the quality of the > product? Maybe anyone can shed light on if the size of the SSD drive > matters? thx will > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Intel P3700 SSD for journals
yes nick, you're right, I can now see on page 16 here www.intel.com/content/www/xa/en/solid-state-drives/ssd-dc-p3700-spec.html there is a difference in the durability. However, I think 7.3PBW isn't much worse than Intel S3610 that's much slower. thx will 400GB: 7.3 PBW 800GB: 14.6 PBW (10 drive writes/day*) 1.6TB: 43.8 PBW (15 drive writes/day*) 2.0TB: 62.05 PBW (17 drive writes/day*) On Fri, Nov 18, 2016 at 9:55 PM, Nick Fisk wrote: > I'm using the 400Gb models as a Journal for 12x drives. I know this is > probably pushing it a little bit, but seems to work fine. I'm > guessing the reason may be relating to the TBW figure being higher on the > more expensive models, maybe they don't want to have to > replace warn NVME's on warranty? > >> -Original Message- >> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of >> William Josefsson >> Sent: 18 November 2016 13:43 >> To: ceph-users >> Subject: [ceph-users] Intel P3700 SSD for journals >> >> Hi list, I wonder if there is anyone who have experience with Intel >> P3700 SSD drives as Journals, and can share their experience? >> >> I was thinking of using the P3700 SSD 400GB as journal in my ceph >> deployment. It is benchmarked in Sebastian hann ssd page as > well. >> However a vendor I spoke to didn't qualify the small sizes of this model as >> "enterprise grade/warranty". They suggested the 1.8TB > or >> 2TB. >> I have asked for clarification on why this is the case. >> >> Has anyone experienced any issues with the smaller size P3700 SSDs, and I'm >> not sure how a smaller drive could affect the quality > of >> the product? Maybe anyone can shed light on if the size of the SSD drive >> matters? thx will >> ___ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Intel P3700 SSD for journals
I'm using the 400Gb models as a Journal for 12x drives. I know this is probably pushing it a little bit, but seems to work fine. I'm guessing the reason may be relating to the TBW figure being higher on the more expensive models, maybe they don't want to have to replace warn NVME's on warranty? > -Original Message- > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of > William Josefsson > Sent: 18 November 2016 13:43 > To: ceph-users > Subject: [ceph-users] Intel P3700 SSD for journals > > Hi list, I wonder if there is anyone who have experience with Intel > P3700 SSD drives as Journals, and can share their experience? > > I was thinking of using the P3700 SSD 400GB as journal in my ceph deployment. > It is benchmarked in Sebastian hann ssd page as well. > However a vendor I spoke to didn't qualify the small sizes of this model as > "enterprise grade/warranty". They suggested the 1.8TB or > 2TB. > I have asked for clarification on why this is the case. > > Has anyone experienced any issues with the smaller size P3700 SSDs, and I'm > not sure how a smaller drive could affect the quality of > the product? Maybe anyone can shed light on if the size of the SSD drive > matters? thx will > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Intel P3700 SSD for journals
Hi list, I wonder if there is anyone who have experience with Intel P3700 SSD drives as Journals, and can share their experience? I was thinking of using the P3700 SSD 400GB as journal in my ceph deployment. It is benchmarked in Sebastian hann ssd page as well. However a vendor I spoke to didn't qualify the small sizes of this model as "enterprise grade/warranty". They suggested the 1.8TB or 2TB. I have asked for clarification on why this is the case. Has anyone experienced any issues with the smaller size P3700 SSDs, and I'm not sure how a smaller drive could affect the quality of the product? Maybe anyone can shed light on if the size of the SSD drive matters? thx will ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
