Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Jan Schermer]

"850 PRO" is a workstation drive. You shouldn't put it in the server...
But it should not just die either way, so don't tell them you use it for Ceph 
next time.

Do the drives work when replugged? Can you get anything from SMART?

Jan


> On 18 Sep 2015, at 02:57, James (Fei) Liu-SSI <james@ssi.samsung.com> 
> wrote:
> 
> Hi Quentin,
> Samsung has so different type of SSD for different type of workload with 
> different SSD media like SLC,MLC,TLC ,3D NAND etc. They were designed for 
> different workloads for different purposes. Thanks for your understanding and 
> support.
>  
> Regards,
> James
>   <>
> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com 
> <mailto:ceph-users-boun...@lists.ceph.com>] On Behalf Of Quentin Hartman
> Sent: Thursday, September 17, 2015 4:05 PM
> To: Andrija Panic
> Cc: ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com>
> Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel 
> s3700
>  
> I ended up having 7 total die. 5 while in service, 2 more when I hooked them 
> up to a test machine to collect information from them. To Samsung's credit, 
> they've been great to deal with and are replacing the failed drives, on the 
> condition that I don't use them for ceph again. Apparently they sent some of 
> my failed drives to an engineer in Korea and they did a failure analysis on 
> them and came to the conclusion they we put to an "unintended use". I have 
> seven left I'm not sure what to do with.
>  
> I've honestly always really liked Samsung, and I'm disappointed that I wasn't 
> able to find anyone with their DC-class drives actually in stock so I ended 
> up switching the to Intel S3700s. My users will be happy to have some SSDs to 
> put in their workstations though!
>  
> QH
>  
> On Thu, Sep 17, 2015 at 4:49 PM, Andrija Panic <andrija.pa...@gmail.com 
> <mailto:andrija.pa...@gmail.com>> wrote:
> Another one bites the dust...
>  
> This is Samsung 850 PRO 256GB... (6 journals on this SSDs just died...)
>  
> [root@cs23 ~]# smartctl -a /dev/sda
> smartctl 5.43 2012-06-30 r3573 [x86_64-linux-3.10.66-1.el6.elrepo.x86_64] 
> (local build)
> Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net 
> <http://smartmontools.sourceforge.net/>
>  
> Vendor:   /1:0:0:0
> Product:
> User Capacity:600,332,565,813,390,450 bytes [600 PB]
> Logical block size:   774843950 bytes
> >> Terminate command early due to bad response to IEC mode page
> A mandatory SMART command failed: exiting. To continue, add one or more '-T 
> permissive' options
>  
> On 8 September 2015 at 18:01, Quentin Hartman <qhart...@direwolfdigital.com 
> <mailto:qhart...@direwolfdigital.com>> wrote:
> On Tue, Sep 8, 2015 at 9:05 AM, Mark Nelson <mnel...@redhat.com 
> <mailto:mnel...@redhat.com>> wrote:
> A list of hardware that is known to work well would be incredibly
> valuable to people getting started. It doesn't have to be exhaustive,
> nor does it have to provide all the guidance someone could want. A
> simple "these things have worked for others" would be sufficient. If
> nothing else, it will help people justify more expensive gear when their
> approval people say "X seems just as good and is cheaper, why can't we
> get that?".
> 
> So I have my opinions on different drives, but I think we do need to be 
> really careful not to appear to endorse or pick on specific vendors. The more 
> we can stick to high-level statements like:
> 
> - Drives should have high write endurance
> - Drives should perform well with O_DSYNC writes
> - Drives should support power loss protection for data in motion
> 
> The better I think.  Once those are established, I think it's reasonable to 
> point out that certain drives meet (or do not meet) those criteria and get 
> feedback from the community as to whether or not vendor's marketing actually 
> reflects reality.  It'd also be really nice to see more information available 
> like the actual hardware (capacitors, flash cells, etc) used in the drives.  
> I've had to show photos of the innards of specific drives to vendors to get 
> them to give me accurate information regarding certain drive capabilities.  
> Having a database of such things available to the community would be really 
> helpful.
> 
>  
> That's probably a very good approach. I think it would be pretty simple to 
> avoid the appearance of endorsement if the data is presented correctly.
>  
> 
> To that point, I think perhaps though something more important than a
> list of known "good" hardware would be a list of known "bad&qu

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

No, they are dead dead dead. Can't get anything off of them. If you look
back further on this thread I think the most noteworthy part of this whole
experience is just how far off my write estimates were. The ones that have
not died have somewhere between 24 and 32 TB written to them after 9 months
in service. This is almost 4x what I thought they would get.

QH

On Fri, Sep 18, 2015 at 1:48 AM, Jan Schermer <j...@schermer.cz> wrote:

> "850 PRO" is a workstation drive. You shouldn't put it in the server...
> But it should not just die either way, so don't tell them you use it for
> Ceph next time.
>
> Do the drives work when replugged? Can you get anything from SMART?
>
> Jan
>
>
> On 18 Sep 2015, at 02:57, James (Fei) Liu-SSI <james@ssi.samsung.com>
> wrote:
>
> Hi Quentin,
> Samsung has so different type of SSD for different type of workload with
> different SSD media like SLC,MLC,TLC ,3D NAND etc. They were designed for
> different workloads for different purposes. Thanks for your understanding
> and support.
>
> Regards,
> James
>
> *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com
> <ceph-users-boun...@lists.ceph.com>] *On Behalf Of *Quentin Hartman
> *Sent:* Thursday, September 17, 2015 4:05 PM
> *To:* Andrija Panic
> *Cc:* ceph-users@lists.ceph.com
> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T vs.
> Intel s3700
>
> I ended up having 7 total die. 5 while in service, 2 more when I hooked
> them up to a test machine to collect information from them. To Samsung's
> credit, they've been great to deal with and are replacing the failed
> drives, on the condition that I don't use them for ceph again. Apparently
> they sent some of my failed drives to an engineer in Korea and they did a
> failure analysis on them and came to the conclusion they we put to an
> "unintended use". I have seven left I'm not sure what to do with.
>
> I've honestly always really liked Samsung, and I'm disappointed that I
> wasn't able to find anyone with their DC-class drives actually in stock so
> I ended up switching the to Intel S3700s. My users will be happy to have
> some SSDs to put in their workstations though!
>
> QH
>
> On Thu, Sep 17, 2015 at 4:49 PM, Andrija Panic <andrija.pa...@gmail.com>
> wrote:
> Another one bites the dust...
>
> This is Samsung 850 PRO 256GB... (6 journals on this SSDs just died...)
>
> [root@cs23 ~]# smartctl -a /dev/sda
> smartctl 5.43 2012-06-30 r3573 [x86_64-linux-3.10.66-1.el6.elrepo.x86_64]
> (local build)
> Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net
>
> Vendor:   /1:0:0:0
> Product:
> User Capacity:600,332,565,813,390,450 bytes [600 PB]
> Logical block size:   774843950 bytes
> >> Terminate command early due to bad response to IEC mode page
> A mandatory SMART command failed: exiting. To continue, add one or more
> '-T permissive' options
>
> On 8 September 2015 at 18:01, Quentin Hartman <
> qhart...@direwolfdigital.com> wrote:
>
> On Tue, Sep 8, 2015 at 9:05 AM, Mark Nelson <mnel...@redhat.com> wrote:
>
> A list of hardware that is known to work well would be incredibly
> valuable to people getting started. It doesn't have to be exhaustive,
> nor does it have to provide all the guidance someone could want. A
> simple "these things have worked for others" would be sufficient. If
> nothing else, it will help people justify more expensive gear when their
> approval people say "X seems just as good and is cheaper, why can't we
> get that?".
>
>
> So I have my opinions on different drives, but I think we do need to be
> really careful not to appear to endorse or pick on specific vendors. The
> more we can stick to high-level statements like:
>
> - Drives should have high write endurance
> - Drives should perform well with O_DSYNC writes
> - Drives should support power loss protection for data in motion
>
> The better I think.  Once those are established, I think it's reasonable
> to point out that certain drives meet (or do not meet) those criteria and
> get feedback from the community as to whether or not vendor's marketing
> actually reflects reality.  It'd also be really nice to see more
> information available like the actual hardware (capacitors, flash cells,
> etc) used in the drives.  I've had to show photos of the innards of
> specific drives to vendors to get them to give me accurate information
> regarding certain drive capabilities.  Having a database of such things
> available to the community would be really helpful.
>
> That's probably a very good approach. I think it would be pretty simple to
> avoid the appearance of endor

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

Another one bites the dust...

This is Samsung 850 PRO 256GB... (6 journals on this SSDs just died...)

[root@cs23 ~]# smartctl -a /dev/sda
smartctl 5.43 2012-06-30 r3573 [x86_64-linux-3.10.66-1.el6.elrepo.x86_64]
(local build)
Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net

Vendor:   /1:0:0:0
Product:
User Capacity:600,332,565,813,390,450 bytes [600 PB]
Logical block size:   774843950 bytes
>> Terminate command early due to bad response to IEC mode page
A mandatory SMART command failed: exiting. To continue, add one or more '-T
permissive' options

On 8 September 2015 at 18:01, Quentin Hartman 
wrote:

> On Tue, Sep 8, 2015 at 9:05 AM, Mark Nelson  wrote:
>
>> A list of hardware that is known to work well would be incredibly
>>> valuable to people getting started. It doesn't have to be exhaustive,
>>> nor does it have to provide all the guidance someone could want. A
>>> simple "these things have worked for others" would be sufficient. If
>>> nothing else, it will help people justify more expensive gear when their
>>> approval people say "X seems just as good and is cheaper, why can't we
>>> get that?".
>>>
>>
>> So I have my opinions on different drives, but I think we do need to be
>> really careful not to appear to endorse or pick on specific vendors. The
>> more we can stick to high-level statements like:
>>
>> - Drives should have high write endurance
>> - Drives should perform well with O_DSYNC writes
>> - Drives should support power loss protection for data in motion
>>
>> The better I think.  Once those are established, I think it's reasonable
>> to point out that certain drives meet (or do not meet) those criteria and
>> get feedback from the community as to whether or not vendor's marketing
>> actually reflects reality.  It'd also be really nice to see more
>> information available like the actual hardware (capacitors, flash cells,
>> etc) used in the drives.  I've had to show photos of the innards of
>> specific drives to vendors to get them to give me accurate information
>> regarding certain drive capabilities.  Having a database of such things
>> available to the community would be really helpful.
>>
>>
> That's probably a very good approach. I think it would be pretty simple to
> avoid the appearance of endorsement if the data is presented correctly.
>
>
>>
>>> To that point, I think perhaps though something more important than a
>>> list of known "good" hardware would be a list of known "bad" hardware,
>>>
>>
>> I'm rather hesitant to do this unless it's been specifically confirmed by
>> the vendor.  It's too easy to point fingers (see the recent kernel trim bug
>> situation).
>
>
> I disagree. I think that only comes into play if you claim to know why the
> hardware has problems. In this case, if you simply state "people who have
> used this drive have experienced a large number of seemingly premature
> failures when using them as journals" that provides sufficient warning to
> users, and if the vendor wants to engage the community and potentially pin
> down why and help us find a way to make the device work or confirm that
> it's just not suited, then that's on them. Samsung seems to be doing
> exactly that. It would be great to have them help provide that level of
> detail, but again, I don't think it's necessary. We're not saying
> "ceph/redhat/$whatever says this hardware sucks" we're saying "The
> community has found that using this hardware with ceph has exhibited these
> negative behaviors...". At that point you're just relaying experiences and
> collecting them in a central location. It's up to the reader to draw
> conclusions from it.
>
> But again, I think more important than either of these would be a
> collection of use cases with actual journal write volumes that have
> occurred in those use cases so that people can make more informed
> purchasing decisions. The fact that my small openstack cluster created 3.6T
> of writes per month on my journal drives (3 OSD each) is somewhat
> mind-blowing. That's almost four times the amount of writes my best guess
> estimates indicated we'd be doing. Clearly there's more going on than we
> are used to paying attention to. Someone coming to ceph and seeing the cost
> of DC-class SSDs versus consumer-class SSDs will almost certainly suffer
> from some amount of sticker shock, and even if they don't their purchasing
> approval people almost certainly will. This is especially true for people
> in smaller organizations where SSDs are still somewhat exotic. And when
> they come back with the "Why won't cheaper thing X be OK?" they need to
> have sufficient information to answer that. Without a test environment to
> generate data with, they will need to rely on the experiences of others,
> and right now those experiences don't seem to be documented anywhere, and
> if they are, they are not very discoverable.
>
> QH
>
> 

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[James (Fei) Liu-SSI]

Hi Quentin,
Samsung has so different type of SSD for different type of workload with 
different SSD media like SLC,MLC,TLC ,3D NAND etc. They were designed for 
different workloads for different purposes. Thanks for your understanding and 
support.

Regards,
James

From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of 
Quentin Hartman
Sent: Thursday, September 17, 2015 4:05 PM
To: Andrija Panic
Cc: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel 
s3700

I ended up having 7 total die. 5 while in service, 2 more when I hooked them up 
to a test machine to collect information from them. To Samsung's credit, 
they've been great to deal with and are replacing the failed drives, on the 
condition that I don't use them for ceph again. Apparently they sent some of my 
failed drives to an engineer in Korea and they did a failure analysis on them 
and came to the conclusion they we put to an "unintended use". I have seven 
left I'm not sure what to do with.

I've honestly always really liked Samsung, and I'm disappointed that I wasn't 
able to find anyone with their DC-class drives actually in stock so I ended up 
switching the to Intel S3700s. My users will be happy to have some SSDs to put 
in their workstations though!

QH

On Thu, Sep 17, 2015 at 4:49 PM, Andrija Panic 
<andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>> wrote:
Another one bites the dust...

This is Samsung 850 PRO 256GB... (6 journals on this SSDs just died...)

[root@cs23 ~]# smartctl -a /dev/sda
smartctl 5.43 2012-06-30 r3573 [x86_64-linux-3.10.66-1.el6.elrepo.x86_64] 
(local build)
Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net

Vendor:   /1:0:0:0
Product:
User Capacity:600,332,565,813,390,450 bytes [600 PB]
Logical block size:   774843950 bytes
>> Terminate command early due to bad response to IEC mode page
A mandatory SMART command failed: exiting. To continue, add one or more '-T 
permissive' options

On 8 September 2015 at 18:01, Quentin Hartman 
<qhart...@direwolfdigital.com<mailto:qhart...@direwolfdigital.com>> wrote:
On Tue, Sep 8, 2015 at 9:05 AM, Mark Nelson 
<mnel...@redhat.com<mailto:mnel...@redhat.com>> wrote:
A list of hardware that is known to work well would be incredibly
valuable to people getting started. It doesn't have to be exhaustive,
nor does it have to provide all the guidance someone could want. A
simple "these things have worked for others" would be sufficient. If
nothing else, it will help people justify more expensive gear when their
approval people say "X seems just as good and is cheaper, why can't we
get that?".

So I have my opinions on different drives, but I think we do need to be really 
careful not to appear to endorse or pick on specific vendors. The more we can 
stick to high-level statements like:

- Drives should have high write endurance
- Drives should perform well with O_DSYNC writes
- Drives should support power loss protection for data in motion

The better I think.  Once those are established, I think it's reasonable to 
point out that certain drives meet (or do not meet) those criteria and get 
feedback from the community as to whether or not vendor's marketing actually 
reflects reality.  It'd also be really nice to see more information available 
like the actual hardware (capacitors, flash cells, etc) used in the drives.  
I've had to show photos of the innards of specific drives to vendors to get 
them to give me accurate information regarding certain drive capabilities.  
Having a database of such things available to the community would be really 
helpful.

That's probably a very good approach. I think it would be pretty simple to 
avoid the appearance of endorsement if the data is presented correctly.


To that point, I think perhaps though something more important than a
list of known "good" hardware would be a list of known "bad" hardware,

I'm rather hesitant to do this unless it's been specifically confirmed by the 
vendor.  It's too easy to point fingers (see the recent kernel trim bug 
situation).

I disagree. I think that only comes into play if you claim to know why the 
hardware has problems. In this case, if you simply state "people who have used 
this drive have experienced a large number of seemingly premature failures when 
using them as journals" that provides sufficient warning to users, and if the 
vendor wants to engage the community and potentially pin down why and help us 
find a way to make the device work or confirm that it's just not suited, then 
that's on them. Samsung seems to be doing exactly that. It would be great to 
have them help provide that level of detail, but again, I don't think it's 
necessary. We're not saying "ceph/redhat/$whatever says this hardware sucks" 
we're saying "The community has found that using

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

"  came to the conclusion they we put to an "unintended use".   "
wtf ? : Best to install them inside shutdown workstation... :)

On 18 September 2015 at 01:04, Quentin Hartman  wrote:

> I ended up having 7 total die. 5 while in service, 2 more when I hooked
> them up to a test machine to collect information from them. To Samsung's
> credit, they've been great to deal with and are replacing the failed
> drives, on the condition that I don't use them for ceph again. Apparently
> they sent some of my failed drives to an engineer in Korea and they did a
> failure analysis on them and came to the conclusion they we put to an
> "unintended use". I have seven left I'm not sure what to do with.
>
> I've honestly always really liked Samsung, and I'm disappointed that I
> wasn't able to find anyone with their DC-class drives actually in stock so
> I ended up switching the to Intel S3700s. My users will be happy to have
> some SSDs to put in their workstations though!
>
> QH
>
> On Thu, Sep 17, 2015 at 4:49 PM, Andrija Panic 
> wrote:
>
>> Another one bites the dust...
>>
>> This is Samsung 850 PRO 256GB... (6 journals on this SSDs just died...)
>>
>> [root@cs23 ~]# smartctl -a /dev/sda
>> smartctl 5.43 2012-06-30 r3573 [x86_64-linux-3.10.66-1.el6.elrepo.x86_64]
>> (local build)
>> Copyright (C) 2002-12 by Bruce Allen,
>> http://smartmontools.sourceforge.net
>>
>> Vendor:   /1:0:0:0
>> Product:
>> User Capacity:600,332,565,813,390,450 bytes [600 PB]
>> Logical block size:   774843950 bytes
>> >> Terminate command early due to bad response to IEC mode page
>> A mandatory SMART command failed: exiting. To continue, add one or more
>> '-T permissive' options
>>
>> On 8 September 2015 at 18:01, Quentin Hartman <
>> qhart...@direwolfdigital.com> wrote:
>>
>>> On Tue, Sep 8, 2015 at 9:05 AM, Mark Nelson  wrote:
>>>
 A list of hardware that is known to work well would be incredibly
> valuable to people getting started. It doesn't have to be exhaustive,
> nor does it have to provide all the guidance someone could want. A
> simple "these things have worked for others" would be sufficient. If
> nothing else, it will help people justify more expensive gear when
> their
> approval people say "X seems just as good and is cheaper, why can't we
> get that?".
>

 So I have my opinions on different drives, but I think we do need to be
 really careful not to appear to endorse or pick on specific vendors. The
 more we can stick to high-level statements like:

 - Drives should have high write endurance
 - Drives should perform well with O_DSYNC writes
 - Drives should support power loss protection for data in motion

 The better I think.  Once those are established, I think it's
 reasonable to point out that certain drives meet (or do not meet) those
 criteria and get feedback from the community as to whether or not vendor's
 marketing actually reflects reality.  It'd also be really nice to see more
 information available like the actual hardware (capacitors, flash cells,
 etc) used in the drives.  I've had to show photos of the innards of
 specific drives to vendors to get them to give me accurate information
 regarding certain drive capabilities.  Having a database of such things
 available to the community would be really helpful.


>>> That's probably a very good approach. I think it would be pretty simple
>>> to avoid the appearance of endorsement if the data is presented correctly.
>>>
>>>

> To that point, I think perhaps though something more important than a
> list of known "good" hardware would be a list of known "bad" hardware,
>

 I'm rather hesitant to do this unless it's been specifically confirmed
 by the vendor.  It's too easy to point fingers (see the recent kernel trim
 bug situation).
>>>
>>>
>>> I disagree. I think that only comes into play if you claim to know why
>>> the hardware has problems. In this case, if you simply state "people who
>>> have used this drive have experienced a large number of seemingly premature
>>> failures when using them as journals" that provides sufficient warning to
>>> users, and if the vendor wants to engage the community and potentially pin
>>> down why and help us find a way to make the device work or confirm that
>>> it's just not suited, then that's on them. Samsung seems to be doing
>>> exactly that. It would be great to have them help provide that level of
>>> detail, but again, I don't think it's necessary. We're not saying
>>> "ceph/redhat/$whatever says this hardware sucks" we're saying "The
>>> community has found that using this hardware with ceph has exhibited these
>>> negative behaviors...". At that point you're just relaying experiences and
>>> collecting them in a central location. It's up to the reader to draw
>>> 

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

I ended up having 7 total die. 5 while in service, 2 more when I hooked
them up to a test machine to collect information from them. To Samsung's
credit, they've been great to deal with and are replacing the failed
drives, on the condition that I don't use them for ceph again. Apparently
they sent some of my failed drives to an engineer in Korea and they did a
failure analysis on them and came to the conclusion they we put to an
"unintended use". I have seven left I'm not sure what to do with.

I've honestly always really liked Samsung, and I'm disappointed that I
wasn't able to find anyone with their DC-class drives actually in stock so
I ended up switching the to Intel S3700s. My users will be happy to have
some SSDs to put in their workstations though!

QH

On Thu, Sep 17, 2015 at 4:49 PM, Andrija Panic 
wrote:

> Another one bites the dust...
>
> This is Samsung 850 PRO 256GB... (6 journals on this SSDs just died...)
>
> [root@cs23 ~]# smartctl -a /dev/sda
> smartctl 5.43 2012-06-30 r3573 [x86_64-linux-3.10.66-1.el6.elrepo.x86_64]
> (local build)
> Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net
>
> Vendor:   /1:0:0:0
> Product:
> User Capacity:600,332,565,813,390,450 bytes [600 PB]
> Logical block size:   774843950 bytes
> >> Terminate command early due to bad response to IEC mode page
> A mandatory SMART command failed: exiting. To continue, add one or more
> '-T permissive' options
>
> On 8 September 2015 at 18:01, Quentin Hartman <
> qhart...@direwolfdigital.com> wrote:
>
>> On Tue, Sep 8, 2015 at 9:05 AM, Mark Nelson  wrote:
>>
>>> A list of hardware that is known to work well would be incredibly
 valuable to people getting started. It doesn't have to be exhaustive,
 nor does it have to provide all the guidance someone could want. A
 simple "these things have worked for others" would be sufficient. If
 nothing else, it will help people justify more expensive gear when their
 approval people say "X seems just as good and is cheaper, why can't we
 get that?".

>>>
>>> So I have my opinions on different drives, but I think we do need to be
>>> really careful not to appear to endorse or pick on specific vendors. The
>>> more we can stick to high-level statements like:
>>>
>>> - Drives should have high write endurance
>>> - Drives should perform well with O_DSYNC writes
>>> - Drives should support power loss protection for data in motion
>>>
>>> The better I think.  Once those are established, I think it's reasonable
>>> to point out that certain drives meet (or do not meet) those criteria and
>>> get feedback from the community as to whether or not vendor's marketing
>>> actually reflects reality.  It'd also be really nice to see more
>>> information available like the actual hardware (capacitors, flash cells,
>>> etc) used in the drives.  I've had to show photos of the innards of
>>> specific drives to vendors to get them to give me accurate information
>>> regarding certain drive capabilities.  Having a database of such things
>>> available to the community would be really helpful.
>>>
>>>
>> That's probably a very good approach. I think it would be pretty simple
>> to avoid the appearance of endorsement if the data is presented correctly.
>>
>>
>>>
 To that point, I think perhaps though something more important than a
 list of known "good" hardware would be a list of known "bad" hardware,

>>>
>>> I'm rather hesitant to do this unless it's been specifically confirmed
>>> by the vendor.  It's too easy to point fingers (see the recent kernel trim
>>> bug situation).
>>
>>
>> I disagree. I think that only comes into play if you claim to know why
>> the hardware has problems. In this case, if you simply state "people who
>> have used this drive have experienced a large number of seemingly premature
>> failures when using them as journals" that provides sufficient warning to
>> users, and if the vendor wants to engage the community and potentially pin
>> down why and help us find a way to make the device work or confirm that
>> it's just not suited, then that's on them. Samsung seems to be doing
>> exactly that. It would be great to have them help provide that level of
>> detail, but again, I don't think it's necessary. We're not saying
>> "ceph/redhat/$whatever says this hardware sucks" we're saying "The
>> community has found that using this hardware with ceph has exhibited these
>> negative behaviors...". At that point you're just relaying experiences and
>> collecting them in a central location. It's up to the reader to draw
>> conclusions from it.
>>
>> But again, I think more important than either of these would be a
>> collection of use cases with actual journal write volumes that have
>> occurred in those use cases so that people can make more informed
>> purchasing decisions. The fact that my small openstack cluster created 3.6T
>> of writes per month on my journal 

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

Well, if you look at the very very fine print on their warranty statement
and some spec sheets they say they are only supposed to be used in "Client
PCs" and if the application exceeds certain write amounts per day, even if
it's below the total volume of writes the drive is supposed to handle, it
voids the warranty. I expect it's the write rate that is killing them.
Purely by measure of the amount of writes, mine should have been at about
50% life or better.

So, according to the strict letter of their specs, a ceph server would be
an unintended use. Of course, all that detail gets omitted in lots of
places where one would do research. In the end though, they are taking care
of me, and frankly that means a lot more in my book. And for what it's
worth, I have many drives from them in PCs and laptops that have been
rolling happily along for years.

QH

On Thu, Sep 17, 2015 at 5:07 PM, Andrija Panic 
wrote:

> "  came to the conclusion they we put to an "unintended use".   "
> wtf ? : Best to install them inside shutdown workstation... :)
>
> On 18 September 2015 at 01:04, Quentin Hartman <
> qhart...@direwolfdigital.com> wrote:
>
>> I ended up having 7 total die. 5 while in service, 2 more when I hooked
>> them up to a test machine to collect information from them. To Samsung's
>> credit, they've been great to deal with and are replacing the failed
>> drives, on the condition that I don't use them for ceph again. Apparently
>> they sent some of my failed drives to an engineer in Korea and they did a
>> failure analysis on them and came to the conclusion they we put to an
>> "unintended use". I have seven left I'm not sure what to do with.
>>
>> I've honestly always really liked Samsung, and I'm disappointed that I
>> wasn't able to find anyone with their DC-class drives actually in stock so
>> I ended up switching the to Intel S3700s. My users will be happy to have
>> some SSDs to put in their workstations though!
>>
>> QH
>>
>> On Thu, Sep 17, 2015 at 4:49 PM, Andrija Panic 
>> wrote:
>>
>>> Another one bites the dust...
>>>
>>> This is Samsung 850 PRO 256GB... (6 journals on this SSDs just died...)
>>>
>>> [root@cs23 ~]# smartctl -a /dev/sda
>>> smartctl 5.43 2012-06-30 r3573
>>> [x86_64-linux-3.10.66-1.el6.elrepo.x86_64] (local build)
>>> Copyright (C) 2002-12 by Bruce Allen,
>>> http://smartmontools.sourceforge.net
>>>
>>> Vendor:   /1:0:0:0
>>> Product:
>>> User Capacity:600,332,565,813,390,450 bytes [600 PB]
>>> Logical block size:   774843950 bytes
>>> >> Terminate command early due to bad response to IEC mode page
>>> A mandatory SMART command failed: exiting. To continue, add one or more
>>> '-T permissive' options
>>>
>>> On 8 September 2015 at 18:01, Quentin Hartman <
>>> qhart...@direwolfdigital.com> wrote:
>>>
 On Tue, Sep 8, 2015 at 9:05 AM, Mark Nelson  wrote:

> A list of hardware that is known to work well would be incredibly
>> valuable to people getting started. It doesn't have to be exhaustive,
>> nor does it have to provide all the guidance someone could want. A
>> simple "these things have worked for others" would be sufficient. If
>> nothing else, it will help people justify more expensive gear when
>> their
>> approval people say "X seems just as good and is cheaper, why can't we
>> get that?".
>>
>
> So I have my opinions on different drives, but I think we do need to
> be really careful not to appear to endorse or pick on specific vendors. 
> The
> more we can stick to high-level statements like:
>
> - Drives should have high write endurance
> - Drives should perform well with O_DSYNC writes
> - Drives should support power loss protection for data in motion
>
> The better I think.  Once those are established, I think it's
> reasonable to point out that certain drives meet (or do not meet) those
> criteria and get feedback from the community as to whether or not vendor's
> marketing actually reflects reality.  It'd also be really nice to see more
> information available like the actual hardware (capacitors, flash cells,
> etc) used in the drives.  I've had to show photos of the innards of
> specific drives to vendors to get them to give me accurate information
> regarding certain drive capabilities.  Having a database of such things
> available to the community would be really helpful.
>
>
 That's probably a very good approach. I think it would be pretty simple
 to avoid the appearance of endorsement if the data is presented correctly.


>
>> To that point, I think perhaps though something more important than a
>> list of known "good" hardware would be a list of known "bad" hardware,
>>
>
> I'm rather hesitant to do this unless it's been specifically confirmed
> by the vendor.  It's too easy to point fingers (see the 

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

On Tue, Sep 8, 2015 at 9:05 AM, Mark Nelson  wrote:

> A list of hardware that is known to work well would be incredibly
>> valuable to people getting started. It doesn't have to be exhaustive,
>> nor does it have to provide all the guidance someone could want. A
>> simple "these things have worked for others" would be sufficient. If
>> nothing else, it will help people justify more expensive gear when their
>> approval people say "X seems just as good and is cheaper, why can't we
>> get that?".
>>
>
> So I have my opinions on different drives, but I think we do need to be
> really careful not to appear to endorse or pick on specific vendors. The
> more we can stick to high-level statements like:
>
> - Drives should have high write endurance
> - Drives should perform well with O_DSYNC writes
> - Drives should support power loss protection for data in motion
>
> The better I think.  Once those are established, I think it's reasonable
> to point out that certain drives meet (or do not meet) those criteria and
> get feedback from the community as to whether or not vendor's marketing
> actually reflects reality.  It'd also be really nice to see more
> information available like the actual hardware (capacitors, flash cells,
> etc) used in the drives.  I've had to show photos of the innards of
> specific drives to vendors to get them to give me accurate information
> regarding certain drive capabilities.  Having a database of such things
> available to the community would be really helpful.
>
>
That's probably a very good approach. I think it would be pretty simple to
avoid the appearance of endorsement if the data is presented correctly.


>
>> To that point, I think perhaps though something more important than a
>> list of known "good" hardware would be a list of known "bad" hardware,
>>
>
> I'm rather hesitant to do this unless it's been specifically confirmed by
> the vendor.  It's too easy to point fingers (see the recent kernel trim bug
> situation).


I disagree. I think that only comes into play if you claim to know why the
hardware has problems. In this case, if you simply state "people who have
used this drive have experienced a large number of seemingly premature
failures when using them as journals" that provides sufficient warning to
users, and if the vendor wants to engage the community and potentially pin
down why and help us find a way to make the device work or confirm that
it's just not suited, then that's on them. Samsung seems to be doing
exactly that. It would be great to have them help provide that level of
detail, but again, I don't think it's necessary. We're not saying
"ceph/redhat/$whatever says this hardware sucks" we're saying "The
community has found that using this hardware with ceph has exhibited these
negative behaviors...". At that point you're just relaying experiences and
collecting them in a central location. It's up to the reader to draw
conclusions from it.

But again, I think more important than either of these would be a
collection of use cases with actual journal write volumes that have
occurred in those use cases so that people can make more informed
purchasing decisions. The fact that my small openstack cluster created 3.6T
of writes per month on my journal drives (3 OSD each) is somewhat
mind-blowing. That's almost four times the amount of writes my best guess
estimates indicated we'd be doing. Clearly there's more going on than we
are used to paying attention to. Someone coming to ceph and seeing the cost
of DC-class SSDs versus consumer-class SSDs will almost certainly suffer
from some amount of sticker shock, and even if they don't their purchasing
approval people almost certainly will. This is especially true for people
in smaller organizations where SSDs are still somewhat exotic. And when
they come back with the "Why won't cheaper thing X be OK?" they need to
have sufficient information to answer that. Without a test environment to
generate data with, they will need to rely on the experiences of others,
and right now those experiences don't seem to be documented anywhere, and
if they are, they are not very discoverable.

QH
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Mark Nelson]

e listed performing total crap (for Journaling)
...but yes, I vote for having some oficial page if possible !

On 7 September 2015 at 11:12, Eino Tuominen <e...@utu.fi
<mailto:e...@utu.fi>> wrote:

Hello,

Should we (somebody, please?) gather up a comprehensive list
of suitable SSD devices to use as ceph journals? This seems to
be a FAQ, and it would be nice if all the knowledge and user
experiences from several different threads could be referenced
easily in the future. I took a look at wiki.ceph.org
<http://wiki.ceph.org/> and there was nothing on this.

--
  Eino Tuominen

-Original Message-
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com
<mailto:ceph-users-boun...@lists.ceph.com>] On Behalf Of Jan
Schermer
Sent: 7. syyskuuta 2015 11:44
        To: Christian Balzer
    Cc: ceph-users; Межов Игорь Александрович
Subject: Re: [ceph-users] which SSD / experiences with Samsung
843T vs. Intel s3700

Re: Samsungs - I feel some of you are mixing and confusing
different Samsung drives.

There is a DC line of Samsung drives meant for DataCenter use.
Those have EVO (write once read many) and PRO (write mostly)
variants.
You don't want to go anywhere near the EVO line with Ceph.
Then there are "regular" EVO and PRO drives - they are not
meant for server use so don't use them.

The main difference is that the "DC" line should provide
reliable and stable performance over time, no surprises, while
the desktop drives can just pause and perform garbage
collection and have completely different cache setup. If you
torture desktop drive hard enough it will protect itself (slow
down to a crawl).

So the only usable drivess for us are "DC PRO" and nothing else.

Jan

> On 05 Sep 2015, at 04:36, Christian Balzer <ch...@gol.com
<mailto:ch...@gol.com>> wrote:
>
>
> Hello,
>
> On Fri, 4 Sep 2015 22:37:06 + Межов Игорь Александрович
wrote:
>
>> Hi!
>>
>>
>> Have worked with Intel DC S3700 200Gb. Due to budget
restrictions, one
>>
>> ssd hosts a system volume and 1:12 OSD journals. 6 nodes,
120Tb raw
>> space.
>>
> Meaning you're limited to 360MB/s writes per node at best.
> But yes, I do understand budget constraints. ^o^
>
>> Cluster serves as RBD storage for ~100VM.
>>
>>
>> Not a  single failure per year - all devices are healthy.
>>
>> The remainig resource (by smart) is ~92%.
>>
> I use 1:2 or 1:3 journals and haven't made any dent into my
200GB S3700
> yet.
>
>>
>> Now we're try to use DC S3710 for journals.
>
> As I wrote a few days ago, unless you go for the 400GB
version the the
> 200GB S3710 is actually slower (for journal purposes) than
the 3700, as
> sequential write speed is the key factor here.
>
> Christian
> --
> Christian BalzerNetwork/Systems Engineer
> ch...@gol.com <mailto:ch...@gol.com> Global OnLine
Japan/Fusion Communications
> http://www.gol.com/
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com>
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




--

Andrija Panić
___
ceph-users mailing list
ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



___
ceph-users mailing list
ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Eino Tuominen]

Hello,

Should we (somebody, please?) gather up a comprehensive list of suitable SSD 
devices to use as ceph journals? This seems to be a FAQ, and it would be nice 
if all the knowledge and user experiences from several different threads could 
be referenced easily in the future. I took a look at wiki.ceph.org and there 
was nothing on this.

-- 
  Eino Tuominen

-Original Message-
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Jan 
Schermer
Sent: 7. syyskuuta 2015 11:44
To: Christian Balzer
Cc: ceph-users; Межов Игорь Александрович
Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel 
s3700

Re: Samsungs - I feel some of you are mixing and confusing different Samsung 
drives.

There is a DC line of Samsung drives meant for DataCenter use. Those have EVO 
(write once read many) and PRO (write mostly) variants.
You don't want to go anywhere near the EVO line with Ceph.
Then there are "regular" EVO and PRO drives - they are not meant for server use 
so don't use them.

The main difference is that the "DC" line should provide reliable and stable 
performance over time, no surprises, while the desktop drives can just pause 
and perform garbage collection and have completely different cache setup. If 
you torture desktop drive hard enough it will protect itself (slow down to a 
crawl).

So the only usable drivess for us are "DC PRO" and nothing else.

Jan

> On 05 Sep 2015, at 04:36, Christian Balzer <ch...@gol.com> wrote:
> 
> 
> Hello,
> 
> On Fri, 4 Sep 2015 22:37:06 + Межов Игорь Александрович wrote:
> 
>> Hi!
>> 
>> 
>> Have worked with Intel DC S3700 200Gb. Due to budget restrictions, one
>> 
>> ssd hosts a system volume and 1:12 OSD journals. 6 nodes, 120Tb raw
>> space.
>> 
> Meaning you're limited to 360MB/s writes per node at best.
> But yes, I do understand budget constraints. ^o^
> 
>> Cluster serves as RBD storage for ~100VM.
>> 
>> 
>> Not a  single failure per year - all devices are healthy.
>> 
>> The remainig resource (by smart) is ~92%.
>> 
> I use 1:2 or 1:3 journals and haven't made any dent into my 200GB S3700
> yet.
> 
>> 
>> Now we're try to use DC S3710 for journals.
> 
> As I wrote a few days ago, unless you go for the 400GB version the the
> 200GB S3710 is actually slower (for journal purposes) than the 3700, as
> sequential write speed is the key factor here.
> 
> Christian
> -- 
> Christian BalzerNetwork/Systems Engineer
> ch...@gol.com Global OnLine Japan/Fusion Communications
> http://www.gol.com/
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

There is
http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

On the other hand, I'm not sure if SSD vendors would be happy to see their
device listed performing total crap (for Journaling) ...but yes, I vote for
having some oficial page if possible !

On 7 September 2015 at 11:12, Eino Tuominen <e...@utu.fi> wrote:

> Hello,
>
> Should we (somebody, please?) gather up a comprehensive list of suitable
> SSD devices to use as ceph journals? This seems to be a FAQ, and it would
> be nice if all the knowledge and user experiences from several different
> threads could be referenced easily in the future. I took a look at
> wiki.ceph.org and there was nothing on this.
>
> --
>   Eino Tuominen
>
> -Original Message-
> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
> Jan Schermer
> Sent: 7. syyskuuta 2015 11:44
> To: Christian Balzer
> Cc: ceph-users; Межов Игорь Александрович
> Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs.
> Intel s3700
>
> Re: Samsungs - I feel some of you are mixing and confusing different
> Samsung drives.
>
> There is a DC line of Samsung drives meant for DataCenter use. Those have
> EVO (write once read many) and PRO (write mostly) variants.
> You don't want to go anywhere near the EVO line with Ceph.
> Then there are "regular" EVO and PRO drives - they are not meant for
> server use so don't use them.
>
> The main difference is that the "DC" line should provide reliable and
> stable performance over time, no surprises, while the desktop drives can
> just pause and perform garbage collection and have completely different
> cache setup. If you torture desktop drive hard enough it will protect
> itself (slow down to a crawl).
>
> So the only usable drivess for us are "DC PRO" and nothing else.
>
> Jan
>
> > On 05 Sep 2015, at 04:36, Christian Balzer <ch...@gol.com> wrote:
> >
> >
> > Hello,
> >
> > On Fri, 4 Sep 2015 22:37:06 + Межов Игорь Александрович wrote:
> >
> >> Hi!
> >>
> >>
> >> Have worked with Intel DC S3700 200Gb. Due to budget restrictions, one
> >>
> >> ssd hosts a system volume and 1:12 OSD journals. 6 nodes, 120Tb raw
> >> space.
> >>
> > Meaning you're limited to 360MB/s writes per node at best.
> > But yes, I do understand budget constraints. ^o^
> >
> >> Cluster serves as RBD storage for ~100VM.
> >>
> >>
> >> Not a  single failure per year - all devices are healthy.
> >>
> >> The remainig resource (by smart) is ~92%.
> >>
> > I use 1:2 or 1:3 journals and haven't made any dent into my 200GB S3700
> > yet.
> >
> >>
> >> Now we're try to use DC S3710 for journals.
> >
> > As I wrote a few days ago, unless you go for the 400GB version the the
> > 200GB S3710 is actually slower (for journal purposes) than the 3700, as
> > sequential write speed is the key factor here.
> >
> > Christian
> > --
> > Christian BalzerNetwork/Systems Engineer
> > ch...@gol.com Global OnLine Japan/Fusion Communications
> > http://www.gol.com/
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>



-- 

Andrija Panić
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

fwiw, I am not confused about the various types of SSDs that Samsung
offers. I knew exactly what I was getting when I ordered them. Based on
their specs and my WAG on how much writing I would be doing they should
have lasted about 6 years. Turns out my estimates were wrong, but even
adjusting for actual use, I should have gotten about 18 months out of these
drives, but I have them dying now at 9 months, with about half of their
theoretical life left.

A list of hardware that is known to work well would be incredibly valuable
to people getting started. It doesn't have to be exhaustive, nor does it
have to provide all the guidance someone could want. A simple "these things
have worked for others" would be sufficient. If nothing else, it will help
people justify more expensive gear when their approval people say "X seems
just as good and is cheaper, why can't we get that?".

To that point, I think perhaps though something more important than a list
of known "good" hardware would be a list of known "bad" hardware, and
perhaps some more experience about what kind of write volume people should
reasonably expect. Setting aside for a moment the early death problem the
recent Samsung drives clearly have (I wonder if it's a side-effect of the
"3D-NAND" tech?) I wouldn't have gotten them had my estimates told me I'd
only get 18 months out of them. That would have also provided me the
information I needed to justify the DC-class drives that cost four times as
much to those that approve purchases. Without that critical piece of
information, I'm left trying to justify thousands of extra dollars with
only "because they're better".

Also, I talked to a Samsung rep last week and he told me the DC 845 line
has been discontinued. The DC-class drives from Samsung are now model
pm863. They are theoretically on the market, but I've not been able to find
them in stock anywhere.

QH

On Mon, Sep 7, 2015 at 4:22 AM, Jan Schermer <j...@schermer.cz> wrote:

> It is not just a question of which SSD.
> It's the combination of distribution (kernel version), disk controller and
> firmware, SSD revision and firmware.
>
> There are several ways to select hardware
> 1) the most traditional way where you build your BoM on a single vendor -
> so you buy servers including SSDs and HBAs as a single unit and then scream
> at the vendor when it doesn't work. I had a good experience with vendors in
> this scenario.
> 2) based on Hardware Compatibility Lists - usually means you can't use tha
> latest hardware. For example LSI doesn't list most SSDs as compatible, or
> they only list really old firmware versions. Unusable, nobody will really
> help you.
> 3) You get a sample and test it, and you hope you will get the same
> hardware when you order in bulk later. We went this route and got nothing
> but trouble when Kingston changed their SSDs completely without changing
> their PN.
>
> Would we recommend s3700/3710 for Ceph? Absolutely. But there are still
> people who have trouble with them in combination with LSI controllers.
> Can we recommend Samsung 845 DC PRO then? I can say it worked nicely with
> my hardware. But surely some people had trouble with it.
>
> I "vote" against creating such a list because of all those reasons, it
> could get someone in trouble.
>
> Jan
>
>
> On 07 Sep 2015, at 11:14, Andrija Panic <andrija.pa...@gmail.com> wrote:
>
> There is
> http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>
> On the other hand, I'm not sure if SSD vendors would be happy to see their
> device listed performing total crap (for Journaling) ...but yes, I vote for
> having some oficial page if possible !
>
> On 7 September 2015 at 11:12, Eino Tuominen <e...@utu.fi> wrote:
>
>> Hello,
>>
>> Should we (somebody, please?) gather up a comprehensive list of suitable
>> SSD devices to use as ceph journals? This seems to be a FAQ, and it would
>> be nice if all the knowledge and user experiences from several different
>> threads could be referenced easily in the future. I took a look at
>> wiki.ceph.org and there was nothing on this.
>>
>> --
>>   Eino Tuominen
>>
>> -Original Message-----
>> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
>> Jan Schermer
>> Sent: 7. syyskuuta 2015 11:44
>> To: Christian Balzer
>> Cc: ceph-users; Межов Игорь Александрович
>> Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs.
>> Intel s3700
>>
>> Re: Samsungs - I feel some of you are mixing and confusing different
>> Samsung drives.
>>
>> There is a DC line of Samsung drives meant for DataCenter use. Those have
>&g

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Jan Schermer]

It is not just a question of which SSD.
It's the combination of distribution (kernel version), disk controller and 
firmware, SSD revision and firmware.

There are several ways to select hardware
1) the most traditional way where you build your BoM on a single vendor - so 
you buy servers including SSDs and HBAs as a single unit and then scream at the 
vendor when it doesn't work. I had a good experience with vendors in this 
scenario.
2) based on Hardware Compatibility Lists - usually means you can't use tha 
latest hardware. For example LSI doesn't list most SSDs as compatible, or they 
only list really old firmware versions. Unusable, nobody will really help you.
3) You get a sample and test it, and you hope you will get the same hardware 
when you order in bulk later. We went this route and got nothing but trouble 
when Kingston changed their SSDs completely without changing their PN.

Would we recommend s3700/3710 for Ceph? Absolutely. But there are still people 
who have trouble with them in combination with LSI controllers.
Can we recommend Samsung 845 DC PRO then? I can say it worked nicely with my 
hardware. But surely some people had trouble with it.

I "vote" against creating such a list because of all those reasons, it could 
get someone in trouble. 

Jan


> On 07 Sep 2015, at 11:14, Andrija Panic <andrija.pa...@gmail.com> wrote:
> 
> There is 
> http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>  
> <http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/>
> 
> On the other hand, I'm not sure if SSD vendors would be happy to see their 
> device listed performing total crap (for Journaling) ...but yes, I vote for 
> having some oficial page if possible !
> 
> On 7 September 2015 at 11:12, Eino Tuominen <e...@utu.fi 
> <mailto:e...@utu.fi>> wrote:
> Hello,
> 
> Should we (somebody, please?) gather up a comprehensive list of suitable SSD 
> devices to use as ceph journals? This seems to be a FAQ, and it would be nice 
> if all the knowledge and user experiences from several different threads 
> could be referenced easily in the future. I took a look at wiki.ceph.org 
> <http://wiki.ceph.org/> and there was nothing on this.
> 
> --
>   Eino Tuominen
> 
> -Original Message-
> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com 
> <mailto:ceph-users-boun...@lists.ceph.com>] On Behalf Of Jan Schermer
> Sent: 7. syyskuuta 2015 11:44
> To: Christian Balzer
> Cc: ceph-users; Межов Игорь Александрович
> Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel 
> s3700
> 
> Re: Samsungs - I feel some of you are mixing and confusing different Samsung 
> drives.
> 
> There is a DC line of Samsung drives meant for DataCenter use. Those have EVO 
> (write once read many) and PRO (write mostly) variants.
> You don't want to go anywhere near the EVO line with Ceph.
> Then there are "regular" EVO and PRO drives - they are not meant for server 
> use so don't use them.
> 
> The main difference is that the "DC" line should provide reliable and stable 
> performance over time, no surprises, while the desktop drives can just pause 
> and perform garbage collection and have completely different cache setup. If 
> you torture desktop drive hard enough it will protect itself (slow down to a 
> crawl).
> 
> So the only usable drivess for us are "DC PRO" and nothing else.
> 
> Jan
> 
> > On 05 Sep 2015, at 04:36, Christian Balzer <ch...@gol.com 
> > <mailto:ch...@gol.com>> wrote:
> >
> >
> > Hello,
> >
> > On Fri, 4 Sep 2015 22:37:06 + Межов Игорь Александрович wrote:
> >
> >> Hi!
> >>
> >>
> >> Have worked with Intel DC S3700 200Gb. Due to budget restrictions, one
> >>
> >> ssd hosts a system volume and 1:12 OSD journals. 6 nodes, 120Tb raw
> >> space.
> >>
> > Meaning you're limited to 360MB/s writes per node at best.
> > But yes, I do understand budget constraints. ^o^
> >
> >> Cluster serves as RBD storage for ~100VM.
> >>
> >>
> >> Not a  single failure per year - all devices are healthy.
> >>
> >> The remainig resource (by smart) is ~92%.
> >>
> > I use 1:2 or 1:3 journals and haven't made any dent into my 200GB S3700
> > yet.
> >
> >>
> >> Now we're try to use DC S3710 for journals.
> >
> > As I wrote a few days ago, unless you go for the 400GB version the the
> > 200GB S3710 is actually slower (for journal purposes) than the 3700, as
> > sequential write speed is 

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[James (Fei) Liu-SSI]

Hi Quentin and Andrija,
Thanks so much for reporting the problems with Samsung.

Would be possible to get to know your configuration of your system?  What kind 
of workload are you running?  Do you use Samsung SSD as separate journaling 
disk, right?

Thanks so much.

James

From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of 
Quentin Hartman
Sent: Thursday, September 03, 2015 1:06 PM
To: Andrija Panic
Cc: ceph-users
Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel 
s3700

Yeah, we've ordered some S3700's to replace them already. Should be here early 
next week. Hopefully they arrive before we have multiple nodes die at once and 
can no longer rebalance successfully.

Most of the drives I have are the 850 Pro 128GB (specifically MZ7KE128HMGA)
There are a couple 120GB 850 EVOs in there too, but ironically, none of them 
have pooped out yet.

On Thu, Sep 3, 2015 at 1:58 PM, Andrija Panic 
<andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>> wrote:

I really advise removing the bastards becore they die...no rebalancing hapening 
just temp osd down while replacing journals...

What size and model are yours Samsungs?
On Sep 3, 2015 7:10 PM, "Quentin Hartman" 
<qhart...@direwolfdigital.com<mailto:qhart...@direwolfdigital.com>> wrote:
We also just started having our 850 Pros die one after the other after about 9 
months of service. 3 down, 11 to go... No warning at all, the drive is fine, 
and then it's not even visible to the machine. According to the stats in hdparm 
and the calcs I did they should have had years of life left, so it seems that 
ceph journals definitely do something they do not like, which is not reflected 
in their stats.

QH

On Wed, Aug 26, 2015 at 7:15 AM, 10 minus 
<t10te...@gmail.com<mailto:t10te...@gmail.com>> wrote:
Hi ,
We got a good deal on 843T and we are using it in our Openstack setup ..as 
journals .
They have been running for last six months ... No issues .
When we compared with  Intel SSDs I think it was 3700 they  were shade slower 
for our workload and considerably cheaper.
We did not run any synthetic benchmark since we had a specific use case.
The performance was better than our old setup so it was good enough.
hth


On Tue, Aug 25, 2015 at 12:07 PM, Andrija Panic 
<andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>> wrote:

We have some 850 pro 256gb ssds if anyone interested to buy:)

And also there was new 850 pro firmware that broke peoples disk which was 
revoked later etc... I'm sticking with only vacuum cleaners from Samsung for 
now, maybe... :)
On Aug 25, 2015 12:02 PM, "Voloshanenko Igor" 
<igor.voloshane...@gmail.com<mailto:igor.voloshane...@gmail.com>> wrote:
To be honest, Samsung 850 PRO not 24/7 series... it's something about desktop+ 
series, but anyway - results from this drives - very very bad in any scenario 
acceptable by real life...

Possible 845 PRO more better, but we don't want to experiment anymore... So we 
choose S3500 240G. Yes, it's cheaper than S3700 (about 2x times), and no so 
durable for writes, but we think more better to replace 1 ssd per 1 year than 
to pay double price now.

2015-08-25 12:59 GMT+03:00 Andrija Panic 
<andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>>:

And should I mention that in another CEPH installation we had samsung 850 pro 
128GB and all of 6 ssds died in 2 month period - simply disappear from the 
system, so not wear out...

Never again we buy Samsung :)
On Aug 25, 2015 11:57 AM, "Andrija Panic" 
<andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>> wrote:

First read please:
http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

We are getting 200 IOPS in comparison to Intels3500 18.000 iops - those are  
constant performance numbers, meaning avoiding drives cache and running for 
longer period of time...
Also if checking with FIO you will get better latencies on intel s3500 (model 
tested in our case) along with 20X better IOPS results...

We observed original issue by having high speed at begining of i.e. file 
transfer inside VM, which than halts to zero... We moved journals back to HDDs 
and performans was acceptable...no we are upgrading to intel S3500...

Best
any details on that ?

On Tue, 25 Aug 2015 11:42:47 +0200, Andrija Panic
<andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>> wrote:

> Make sure you test what ever you decide. We just learned this the hard way
> with samsung 850 pro, which is total crap, more than you could imagine...
>
> Andrija
> On Aug 25, 2015 11:25 AM, "Jan Schermer" 
> <j...@schermer.cz<mailto:j...@schermer.cz>> wrote:
>
> > I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
> > Very cheap, better than Intel 3610 for sure (and I think it beats even
> > 37

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

Quentin,

try fio or dd with O_DIRECT and D_SYNC flags, and you will see less than
1MB/s - that is common for most "home" drives - check the post down to
understand

We removed all Samsung 850 pro 256GB from our new CEPH installation and
replaced with Intel S3500 (18.000 (4Kb) IOPS constant write speed with
O_DIRECT, D_SYNC, in comparison to 200 IOPS for Samsun 850pro - you can
imagine the difference...):

http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

Best

On 4 September 2015 at 21:09, Quentin Hartman <qhart...@direwolfdigital.com>
wrote:

> Mine are also mostly 850 Pros. I have a few 840s, and a few 850 EVOs in
> there just because I couldn't find 14 pros at the time we were ordering
> hardware. I have 14 nodes, each with a single 128 or 120GB SSD that serves
> as the boot drive  and the journal for 3 OSDs. And similarly, mine just
> started disappearing a few weeks ago. I've now had four fail (three 850
> Pro, one 840 Pro). I expect the rest to fail any day.
>
> As it turns out I had a phone conversation with the support rep who has
> been helping me with RMA's today and he's putting together a report with my
> pertinent information in it to forward on to someone.
>
> FWIW, I tried to get your 845's for this deploy, but couldn't find them
> anywhere, and since the 850's looked about as durable on paper I figured
> they would do ok. Seems not to be the case.
>
> QH
>
> On Fri, Sep 4, 2015 at 12:53 PM, Andrija Panic <andrija.pa...@gmail.com>
> wrote:
>
>> Hi James,
>>
>> I had 3 CEPH nodes as folowing: 12 OSDs(HDD) and 2 SSDs (2x 6 Journals
>> partitions on each SSD) - SSDs just vanished with no warning, no smartctl
>> errors nothing... so 2 SSDs in 3 servers vanished in...2-3 weeks, after a
>> 3-4 months of being in production (VMs/KVM/CloudStack)
>>
>> Mine were also Samsung 850 PRO 128GB.
>>
>> Best,
>> Andrija
>>
>> On 4 September 2015 at 19:27, James (Fei) Liu-SSI <
>> james@ssi.samsung.com> wrote:
>>
>>> Hi Quentin and Andrija,
>>>
>>> Thanks so much for reporting the problems with Samsung.
>>>
>>>
>>>
>>> Would be possible to get to know your configuration of your system?
>>> What kind of workload are you running?  Do you use Samsung SSD as separate
>>> journaling disk, right?
>>>
>>>
>>>
>>> Thanks so much.
>>>
>>>
>>>
>>> James
>>>
>>>
>>>
>>> *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On
>>> Behalf Of *Quentin Hartman
>>> *Sent:* Thursday, September 03, 2015 1:06 PM
>>> *To:* Andrija Panic
>>> *Cc:* ceph-users
>>> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T
>>> vs. Intel s3700
>>>
>>>
>>>
>>> Yeah, we've ordered some S3700's to replace them already. Should be here
>>> early next week. Hopefully they arrive before we have multiple nodes die at
>>> once and can no longer rebalance successfully.
>>>
>>>
>>>
>>> Most of the drives I have are the 850 Pro 128GB (specifically
>>> MZ7KE128HMGA)
>>>
>>> There are a couple 120GB 850 EVOs in there too, but ironically, none of
>>> them have pooped out yet.
>>>
>>>
>>>
>>> On Thu, Sep 3, 2015 at 1:58 PM, Andrija Panic <andrija.pa...@gmail.com>
>>> wrote:
>>>
>>> I really advise removing the bastards becore they die...no rebalancing
>>> hapening just temp osd down while replacing journals...
>>>
>>> What size and model are yours Samsungs?
>>>
>>> On Sep 3, 2015 7:10 PM, "Quentin Hartman" <qhart...@direwolfdigital.com>
>>> wrote:
>>>
>>> We also just started having our 850 Pros die one after the other after
>>> about 9 months of service. 3 down, 11 to go... No warning at all, the drive
>>> is fine, and then it's not even visible to the machine. According to the
>>> stats in hdparm and the calcs I did they should have had years of life
>>> left, so it seems that ceph journals definitely do something they do not
>>> like, which is not reflected in their stats.
>>>
>>>
>>>
>>> QH
>>>
>>>
>>>
>>> On Wed, Aug 26, 2015 at 7:15 AM, 10 minus <t10te...@gmail.com> wrote:
>>>
>>> Hi ,
>>>
>>> We got a good deal on 843T and we are using it in our Openstack setup
>>> ..as journals .
>>> The

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

Yeah, we've ordered some S3700's since we can't afford to have these sorts
of failures and haven't been able to find any of the DC-rated Samsung
drives anywhere.

fwiw, we didn't have any performance problems with the samsungs, it's
exclusively this sudden failure that's making us look elsewhere.

QH

On Fri, Sep 4, 2015 at 1:20 PM, Andrija Panic <andrija.pa...@gmail.com>
wrote:

> Quentin,
>
> try fio or dd with O_DIRECT and D_SYNC flags, and you will see less than
> 1MB/s - that is common for most "home" drives - check the post down to
> understand
>
> We removed all Samsung 850 pro 256GB from our new CEPH installation and
> replaced with Intel S3500 (18.000 (4Kb) IOPS constant write speed with
> O_DIRECT, D_SYNC, in comparison to 200 IOPS for Samsun 850pro - you can
> imagine the difference...):
>
>
> http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>
> Best
>
> On 4 September 2015 at 21:09, Quentin Hartman <
> qhart...@direwolfdigital.com> wrote:
>
>> Mine are also mostly 850 Pros. I have a few 840s, and a few 850 EVOs in
>> there just because I couldn't find 14 pros at the time we were ordering
>> hardware. I have 14 nodes, each with a single 128 or 120GB SSD that serves
>> as the boot drive  and the journal for 3 OSDs. And similarly, mine just
>> started disappearing a few weeks ago. I've now had four fail (three 850
>> Pro, one 840 Pro). I expect the rest to fail any day.
>>
>> As it turns out I had a phone conversation with the support rep who has
>> been helping me with RMA's today and he's putting together a report with my
>> pertinent information in it to forward on to someone.
>>
>> FWIW, I tried to get your 845's for this deploy, but couldn't find them
>> anywhere, and since the 850's looked about as durable on paper I figured
>> they would do ok. Seems not to be the case.
>>
>> QH
>>
>> On Fri, Sep 4, 2015 at 12:53 PM, Andrija Panic <andrija.pa...@gmail.com>
>> wrote:
>>
>>> Hi James,
>>>
>>> I had 3 CEPH nodes as folowing: 12 OSDs(HDD) and 2 SSDs (2x 6 Journals
>>> partitions on each SSD) - SSDs just vanished with no warning, no smartctl
>>> errors nothing... so 2 SSDs in 3 servers vanished in...2-3 weeks, after a
>>> 3-4 months of being in production (VMs/KVM/CloudStack)
>>>
>>> Mine were also Samsung 850 PRO 128GB.
>>>
>>> Best,
>>> Andrija
>>>
>>> On 4 September 2015 at 19:27, James (Fei) Liu-SSI <
>>> james@ssi.samsung.com> wrote:
>>>
>>>> Hi Quentin and Andrija,
>>>>
>>>> Thanks so much for reporting the problems with Samsung.
>>>>
>>>>
>>>>
>>>> Would be possible to get to know your configuration of your system?
>>>> What kind of workload are you running?  Do you use Samsung SSD as separate
>>>> journaling disk, right?
>>>>
>>>>
>>>>
>>>> Thanks so much.
>>>>
>>>>
>>>>
>>>> James
>>>>
>>>>
>>>>
>>>> *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On
>>>> Behalf Of *Quentin Hartman
>>>> *Sent:* Thursday, September 03, 2015 1:06 PM
>>>> *To:* Andrija Panic
>>>> *Cc:* ceph-users
>>>> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T
>>>> vs. Intel s3700
>>>>
>>>>
>>>>
>>>> Yeah, we've ordered some S3700's to replace them already. Should be
>>>> here early next week. Hopefully they arrive before we have multiple nodes
>>>> die at once and can no longer rebalance successfully.
>>>>
>>>>
>>>>
>>>> Most of the drives I have are the 850 Pro 128GB (specifically
>>>> MZ7KE128HMGA)
>>>>
>>>> There are a couple 120GB 850 EVOs in there too, but ironically, none of
>>>> them have pooped out yet.
>>>>
>>>>
>>>>
>>>> On Thu, Sep 3, 2015 at 1:58 PM, Andrija Panic <andrija.pa...@gmail.com>
>>>> wrote:
>>>>
>>>> I really advise removing the bastards becore they die...no rebalancing
>>>> hapening just temp osd down while replacing journals...
>>>>
>>>> What size and model are yours Samsungs?
>>>>
>>>> On Sep 3, 2015 7:10 PM, "Quentin Hartman" <qhart...@direwolfdigital.com>
>>>> wrote:
>>>>

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

Hi James,

yes CEPH with Cloudstack. all 6 SSDs (2 SSDs in each of 3 nodes) vanished
in 2-3 weeks total time, and yes brand new Samsung 850 Pro 128GB - I also
checked wear_level atribute via smartctl prior to all drives dying - no
indication wear_level is low or anything...also all other parametres seemed
fine...

I cant reproduce setup, we returned all 850 pros...

Hardware configuration: server model (
http://www.quantaqct.com/Product/Servers/Rackmount-Servers/2U/STRATOS-S210-X22RQ-p7c77c70c83c118?search=S210-X22RQ)
= 64GB RAM, 2 x Intel 2620 v2 CPU - 12 HDDS connected from the front of
server to main disk backplain (12 OSDs) and 2 SSDs connected to embedded
Intel C601 controler on back of the servers (6 partitions on each SSD for
Jorunals + 1 partition used for OS)...

As for workload, I dont think we had very heavy workload at all, since not
to many VMs were running there, and it was mostly Linux web servers...

Best,
Andrija

On 4 September 2015 at 21:15, James (Fei) Liu-SSI <james@ssi.samsung.com
> wrote:

> Hi Andrija,
>
> Thanks for your promptly response. Would be possible to have any change to
> know your hardware configuration including your server information?
> Secondly, Is there anyway to duplicate your workload with fio-rbd, rbd
> bench or rados bench?
>
>
>
>   “so 2 SSDs in 3 servers vanished in...2-3 weeks, after a 3-4 months of
> being in production (VMs/KVM/CloudStack)”
>
>
>
>What you mean over here is that you deploy Ceph with CloudStack , am I
> correct? The 2 SSDs vanished in 2~3 weeks is brand new Samsung 850 Pro
> 128GB, right?
>
>
>
> Thanks,
>
> James
>
>
>
> *From:* Andrija Panic [mailto:andrija.pa...@gmail.com]
> *Sent:* Friday, September 04, 2015 11:53 AM
> *To:* James (Fei) Liu-SSI
> *Cc:* Quentin Hartman; ceph-users
>
> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T vs.
> Intel s3700
>
>
>
> Hi James,
>
>
>
> I had 3 CEPH nodes as folowing: 12 OSDs(HDD) and 2 SSDs (2x 6 Journals
> partitions on each SSD) - SSDs just vanished with no warning, no smartctl
> errors nothing... so 2 SSDs in 3 servers vanished in...2-3 weeks, after a
> 3-4 months of being in production (VMs/KVM/CloudStack)
>
> Mine were also Samsung 850 PRO 128GB.
>
>
>
> Best,
>
> Andrija
>
>
>
> On 4 September 2015 at 19:27, James (Fei) Liu-SSI <
> james@ssi.samsung.com> wrote:
>
> Hi Quentin and Andrija,
>
> Thanks so much for reporting the problems with Samsung.
>
>
>
> Would be possible to get to know your configuration of your system?  What
> kind of workload are you running?  Do you use Samsung SSD as separate
> journaling disk, right?
>
>
>
> Thanks so much.
>
>
>
> James
>
>
>
> *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf
> Of *Quentin Hartman
> *Sent:* Thursday, September 03, 2015 1:06 PM
> *To:* Andrija Panic
> *Cc:* ceph-users
> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T vs.
> Intel s3700
>
>
>
> Yeah, we've ordered some S3700's to replace them already. Should be here
> early next week. Hopefully they arrive before we have multiple nodes die at
> once and can no longer rebalance successfully.
>
>
>
> Most of the drives I have are the 850 Pro 128GB (specifically
> MZ7KE128HMGA)
>
> There are a couple 120GB 850 EVOs in there too, but ironically, none of
> them have pooped out yet.
>
>
>
> On Thu, Sep 3, 2015 at 1:58 PM, Andrija Panic <andrija.pa...@gmail.com>
> wrote:
>
> I really advise removing the bastards becore they die...no rebalancing
> hapening just temp osd down while replacing journals...
>
> What size and model are yours Samsungs?
>
> On Sep 3, 2015 7:10 PM, "Quentin Hartman" <qhart...@direwolfdigital.com>
> wrote:
>
> We also just started having our 850 Pros die one after the other after
> about 9 months of service. 3 down, 11 to go... No warning at all, the drive
> is fine, and then it's not even visible to the machine. According to the
> stats in hdparm and the calcs I did they should have had years of life
> left, so it seems that ceph journals definitely do something they do not
> like, which is not reflected in their stats.
>
>
>
> QH
>
>
>
> On Wed, Aug 26, 2015 at 7:15 AM, 10 minus <t10te...@gmail.com> wrote:
>
> Hi ,
>
> We got a good deal on 843T and we are using it in our Openstack setup ..as
> journals .
> They have been running for last six months ... No issues .
>
> When we compared with  Intel SSDs I think it was 3700 they  were shade
> slower for our workload and considerably cheaper.
>
> We did not run any synthetic benchmark since we had 

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[James (Fei) Liu-SSI]

Andrija,
In your email thread, (18.000 (4Kb) IOPS constant write speed stands for 18K 
iops with 4k block size, right? However, you can only achieve 200IOPS with 
Samsung 850Pro, right?

Theoretically, Samsung 850 Pro can get up to 100,000 IOPS with 4k Random Read 
with certain workload.  It is a little bit strange over here.

Regards,
James


From: Andrija Panic [mailto:andrija.pa...@gmail.com]
Sent: Friday, September 04, 2015 12:21 PM
To: Quentin Hartman
Cc: James (Fei) Liu-SSI; ceph-users
Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel 
s3700

Quentin,

try fio or dd with O_DIRECT and D_SYNC flags, and you will see less than 1MB/s 
- that is common for most "home" drives - check the post down to understand
We removed all Samsung 850 pro 256GB from our new CEPH installation and 
replaced with Intel S3500 (18.000 (4Kb) IOPS constant write speed with 
O_DIRECT, D_SYNC, in comparison to 200 IOPS for Samsun 850pro - you can imagine 
the difference...):
http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

Best

On 4 September 2015 at 21:09, Quentin Hartman 
<qhart...@direwolfdigital.com<mailto:qhart...@direwolfdigital.com>> wrote:
Mine are also mostly 850 Pros. I have a few 840s, and a few 850 EVOs in there 
just because I couldn't find 14 pros at the time we were ordering hardware. I 
have 14 nodes, each with a single 128 or 120GB SSD that serves as the boot 
drive  and the journal for 3 OSDs. And similarly, mine just started 
disappearing a few weeks ago. I've now had four fail (three 850 Pro, one 840 
Pro). I expect the rest to fail any day.

As it turns out I had a phone conversation with the support rep who has been 
helping me with RMA's today and he's putting together a report with my 
pertinent information in it to forward on to someone.

FWIW, I tried to get your 845's for this deploy, but couldn't find them 
anywhere, and since the 850's looked about as durable on paper I figured they 
would do ok. Seems not to be the case.

QH

On Fri, Sep 4, 2015 at 12:53 PM, Andrija Panic 
<andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>> wrote:
Hi James,

I had 3 CEPH nodes as folowing: 12 OSDs(HDD) and 2 SSDs (2x 6 Journals 
partitions on each SSD) - SSDs just vanished with no warning, no smartctl 
errors nothing... so 2 SSDs in 3 servers vanished in...2-3 weeks, after a 3-4 
months of being in production (VMs/KVM/CloudStack)

Mine were also Samsung 850 PRO 128GB.

Best,
Andrija

On 4 September 2015 at 19:27, James (Fei) Liu-SSI 
<james@ssi.samsung.com<mailto:james@ssi.samsung.com>> wrote:
Hi Quentin and Andrija,
Thanks so much for reporting the problems with Samsung.

Would be possible to get to know your configuration of your system?  What kind 
of workload are you running?  Do you use Samsung SSD as separate journaling 
disk, right?

Thanks so much.

James

From: ceph-users 
[mailto:ceph-users-boun...@lists.ceph.com<mailto:ceph-users-boun...@lists.ceph.com>]
 On Behalf Of Quentin Hartman
Sent: Thursday, September 03, 2015 1:06 PM
To: Andrija Panic
Cc: ceph-users
Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel 
s3700

Yeah, we've ordered some S3700's to replace them already. Should be here early 
next week. Hopefully they arrive before we have multiple nodes die at once and 
can no longer rebalance successfully.

Most of the drives I have are the 850 Pro 128GB (specifically MZ7KE128HMGA)
There are a couple 120GB 850 EVOs in there too, but ironically, none of them 
have pooped out yet.

On Thu, Sep 3, 2015 at 1:58 PM, Andrija Panic 
<andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>> wrote:

I really advise removing the bastards becore they die...no rebalancing hapening 
just temp osd down while replacing journals...

What size and model are yours Samsungs?
On Sep 3, 2015 7:10 PM, "Quentin Hartman" 
<qhart...@direwolfdigital.com<mailto:qhart...@direwolfdigital.com>> wrote:
We also just started having our 850 Pros die one after the other after about 9 
months of service. 3 down, 11 to go... No warning at all, the drive is fine, 
and then it's not even visible to the machine. According to the stats in hdparm 
and the calcs I did they should have had years of life left, so it seems that 
ceph journals definitely do something they do not like, which is not reflected 
in their stats.

QH

On Wed, Aug 26, 2015 at 7:15 AM, 10 minus 
<t10te...@gmail.com<mailto:t10te...@gmail.com>> wrote:
Hi ,
We got a good deal on 843T and we are using it in our Openstack setup ..as 
journals .
They have been running for last six months ... No issues .
When we compared with  Intel SSDs I think it was 3700 they  were shade slower 
for our workload and considerably cheaper.
We did not run any synthetic benchmark since we had a specific use case.
The performance was better than our old setup so 

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

Mine are also mostly 850 Pros. I have a few 840s, and a few 850 EVOs in
there just because I couldn't find 14 pros at the time we were ordering
hardware. I have 14 nodes, each with a single 128 or 120GB SSD that serves
as the boot drive  and the journal for 3 OSDs. And similarly, mine just
started disappearing a few weeks ago. I've now had four fail (three 850
Pro, one 840 Pro). I expect the rest to fail any day.

As it turns out I had a phone conversation with the support rep who has
been helping me with RMA's today and he's putting together a report with my
pertinent information in it to forward on to someone.

FWIW, I tried to get your 845's for this deploy, but couldn't find them
anywhere, and since the 850's looked about as durable on paper I figured
they would do ok. Seems not to be the case.

QH

On Fri, Sep 4, 2015 at 12:53 PM, Andrija Panic <andrija.pa...@gmail.com>
wrote:

> Hi James,
>
> I had 3 CEPH nodes as folowing: 12 OSDs(HDD) and 2 SSDs (2x 6 Journals
> partitions on each SSD) - SSDs just vanished with no warning, no smartctl
> errors nothing... so 2 SSDs in 3 servers vanished in...2-3 weeks, after a
> 3-4 months of being in production (VMs/KVM/CloudStack)
>
> Mine were also Samsung 850 PRO 128GB.
>
> Best,
> Andrija
>
> On 4 September 2015 at 19:27, James (Fei) Liu-SSI <
> james@ssi.samsung.com> wrote:
>
>> Hi Quentin and Andrija,
>>
>> Thanks so much for reporting the problems with Samsung.
>>
>>
>>
>> Would be possible to get to know your configuration of your system?  What
>> kind of workload are you running?  Do you use Samsung SSD as separate
>> journaling disk, right?
>>
>>
>>
>> Thanks so much.
>>
>>
>>
>> James
>>
>>
>>
>> *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf
>> Of *Quentin Hartman
>> *Sent:* Thursday, September 03, 2015 1:06 PM
>> *To:* Andrija Panic
>> *Cc:* ceph-users
>> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T
>> vs. Intel s3700
>>
>>
>>
>> Yeah, we've ordered some S3700's to replace them already. Should be here
>> early next week. Hopefully they arrive before we have multiple nodes die at
>> once and can no longer rebalance successfully.
>>
>>
>>
>> Most of the drives I have are the 850 Pro 128GB (specifically
>> MZ7KE128HMGA)
>>
>> There are a couple 120GB 850 EVOs in there too, but ironically, none of
>> them have pooped out yet.
>>
>>
>>
>> On Thu, Sep 3, 2015 at 1:58 PM, Andrija Panic <andrija.pa...@gmail.com>
>> wrote:
>>
>> I really advise removing the bastards becore they die...no rebalancing
>> hapening just temp osd down while replacing journals...
>>
>> What size and model are yours Samsungs?
>>
>> On Sep 3, 2015 7:10 PM, "Quentin Hartman" <qhart...@direwolfdigital.com>
>> wrote:
>>
>> We also just started having our 850 Pros die one after the other after
>> about 9 months of service. 3 down, 11 to go... No warning at all, the drive
>> is fine, and then it's not even visible to the machine. According to the
>> stats in hdparm and the calcs I did they should have had years of life
>> left, so it seems that ceph journals definitely do something they do not
>> like, which is not reflected in their stats.
>>
>>
>>
>> QH
>>
>>
>>
>> On Wed, Aug 26, 2015 at 7:15 AM, 10 minus <t10te...@gmail.com> wrote:
>>
>> Hi ,
>>
>> We got a good deal on 843T and we are using it in our Openstack setup
>> ..as journals .
>> They have been running for last six months ... No issues .
>>
>> When we compared with  Intel SSDs I think it was 3700 they  were shade
>> slower for our workload and considerably cheaper.
>>
>> We did not run any synthetic benchmark since we had a specific use case.
>>
>> The performance was better than our old setup so it was good enough.
>>
>> hth
>>
>>
>>
>> On Tue, Aug 25, 2015 at 12:07 PM, Andrija Panic <andrija.pa...@gmail.com>
>> wrote:
>>
>> We have some 850 pro 256gb ssds if anyone interested to buy:)
>>
>> And also there was new 850 pro firmware that broke peoples disk which was
>> revoked later etc... I'm sticking with only vacuum cleaners from Samsung
>> for now, maybe... :)
>>
>> On Aug 25, 2015 12:02 PM, "Voloshanenko Igor" <
>> igor.voloshane...@gmail.com> wrote:
>>
>> To be honest, Samsung 850 PRO not 24/7 series... it's something about
>> desktop+ series, but anyway - results

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

 is still less than I'd calculated when I bought these drives. Though
I assumed a much slower wear rate (1TB / month) than what we're actually
apparently getting (3.6TB / month), so my original estimated lifespan of
about 6 years was way off.

QH


On Fri, Sep 4, 2015 at 1:15 PM, James (Fei) Liu-SSI <
james@ssi.samsung.com> wrote:

> Hi Andrija,
>
> Thanks for your promptly response. Would be possible to have any change to
> know your hardware configuration including your server information?
> Secondly, Is there anyway to duplicate your workload with fio-rbd, rbd
> bench or rados bench?
>
>
>
>   “so 2 SSDs in 3 servers vanished in...2-3 weeks, after a 3-4 months of
> being in production (VMs/KVM/CloudStack)”
>
>
>
>What you mean over here is that you deploy Ceph with CloudStack , am I
> correct? The 2 SSDs vanished in 2~3 weeks is brand new Samsung 850 Pro
> 128GB, right?
>
>
>
> Thanks,
>
> James
>
>
>
> *From:* Andrija Panic [mailto:andrija.pa...@gmail.com]
> *Sent:* Friday, September 04, 2015 11:53 AM
> *To:* James (Fei) Liu-SSI
> *Cc:* Quentin Hartman; ceph-users
>
> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T vs.
> Intel s3700
>
>
>
> Hi James,
>
>
>
> I had 3 CEPH nodes as folowing: 12 OSDs(HDD) and 2 SSDs (2x 6 Journals
> partitions on each SSD) - SSDs just vanished with no warning, no smartctl
> errors nothing... so 2 SSDs in 3 servers vanished in...2-3 weeks, after a
> 3-4 months of being in production (VMs/KVM/CloudStack)
>
> Mine were also Samsung 850 PRO 128GB.
>
>
>
> Best,
>
> Andrija
>
>
>
> On 4 September 2015 at 19:27, James (Fei) Liu-SSI <
> james@ssi.samsung.com> wrote:
>
> Hi Quentin and Andrija,
>
> Thanks so much for reporting the problems with Samsung.
>
>
>
> Would be possible to get to know your configuration of your system?  What
> kind of workload are you running?  Do you use Samsung SSD as separate
> journaling disk, right?
>
>
>
> Thanks so much.
>
>
>
> James
>
>
>
> *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf
> Of *Quentin Hartman
> *Sent:* Thursday, September 03, 2015 1:06 PM
> *To:* Andrija Panic
> *Cc:* ceph-users
> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T vs.
> Intel s3700
>
>
>
> Yeah, we've ordered some S3700's to replace them already. Should be here
> early next week. Hopefully they arrive before we have multiple nodes die at
> once and can no longer rebalance successfully.
>
>
>
> Most of the drives I have are the 850 Pro 128GB (specifically
> MZ7KE128HMGA)
>
> There are a couple 120GB 850 EVOs in there too, but ironically, none of
> them have pooped out yet.
>
>
>
> On Thu, Sep 3, 2015 at 1:58 PM, Andrija Panic <andrija.pa...@gmail.com>
> wrote:
>
> I really advise removing the bastards becore they die...no rebalancing
> hapening just temp osd down while replacing journals...
>
> What size and model are yours Samsungs?
>
> On Sep 3, 2015 7:10 PM, "Quentin Hartman" <qhart...@direwolfdigital.com>
> wrote:
>
> We also just started having our 850 Pros die one after the other after
> about 9 months of service. 3 down, 11 to go... No warning at all, the drive
> is fine, and then it's not even visible to the machine. According to the
> stats in hdparm and the calcs I did they should have had years of life
> left, so it seems that ceph journals definitely do something they do not
> like, which is not reflected in their stats.
>
>
>
> QH
>
>
>
> On Wed, Aug 26, 2015 at 7:15 AM, 10 minus <t10te...@gmail.com> wrote:
>
> Hi ,
>
> We got a good deal on 843T and we are using it in our Openstack setup ..as
> journals .
> They have been running for last six months ... No issues .
>
> When we compared with  Intel SSDs I think it was 3700 they  were shade
> slower for our workload and considerably cheaper.
>
> We did not run any synthetic benchmark since we had a specific use case.
>
> The performance was better than our old setup so it was good enough.
>
> hth
>
>
>
> On Tue, Aug 25, 2015 at 12:07 PM, Andrija Panic <andrija.pa...@gmail.com>
> wrote:
>
> We have some 850 pro 256gb ssds if anyone interested to buy:)
>
> And also there was new 850 pro firmware that broke peoples disk which was
> revoked later etc... I'm sticking with only vacuum cleaners from Samsung
> for now, maybe... :)
>
> On Aug 25, 2015 12:02 PM, "Voloshanenko Igor" <igor.voloshane...@gmail.com>
> wrote:
>
> To be honest, Samsung 850 PRO not 24/7 series... it's something about
> desktop+

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[James (Fei) Liu-SSI]

Hi Anrija,
Your feedback is greatly appreciated.

Regards,
James

From: Andrija Panic [mailto:andrija.pa...@gmail.com]
Sent: Friday, September 04, 2015 12:39 PM
To: James (Fei) Liu-SSI
Cc: Quentin Hartman; ceph-users
Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel 
s3700

James,

there are simple FIO tests or even DD test on Linux, which you can run to see 
how good SSD will perform as CEPH Journal device (CEPH does writes with 
O_DIRECT and D_SYNC flags to SSDs) - Samsung 850 perform here extremely bad, as 
many, many other vendors (D_SYNC kills performance for them...)

If you are not using D_SYNC flag, then Samsung can achieve some nice numbers...
dd if=/dev/zero of=/dev/sda bs=4k count=10 oflag=direct,dsync (where 
/dev/sda is raw drive, or replace that with mount point i.e. /root/ddfile)

Check post for more info please: 
http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
Thanks

On 4 September 2015 at 21:31, James (Fei) Liu-SSI 
<james@ssi.samsung.com<mailto:james@ssi.samsung.com>> wrote:
Andrija,
In your email thread, (18.000 (4Kb) IOPS constant write speed stands for 18K 
iops with 4k block size, right? However, you can only achieve 200IOPS with 
Samsung 850Pro, right?

Theoretically, Samsung 850 Pro can get up to 100,000 IOPS with 4k Random Read 
with certain workload.  It is a little bit strange over here.

Regards,
James


From: Andrija Panic 
[mailto:andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>]
Sent: Friday, September 04, 2015 12:21 PM
To: Quentin Hartman
Cc: James (Fei) Liu-SSI; ceph-users

Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel 
s3700

Quentin,

try fio or dd with O_DIRECT and D_SYNC flags, and you will see less than 1MB/s 
- that is common for most "home" drives - check the post down to understand
We removed all Samsung 850 pro 256GB from our new CEPH installation and 
replaced with Intel S3500 (18.000 (4Kb) IOPS constant write speed with 
O_DIRECT, D_SYNC, in comparison to 200 IOPS for Samsun 850pro - you can imagine 
the difference...):
http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

Best

On 4 September 2015 at 21:09, Quentin Hartman 
<qhart...@direwolfdigital.com<mailto:qhart...@direwolfdigital.com>> wrote:
Mine are also mostly 850 Pros. I have a few 840s, and a few 850 EVOs in there 
just because I couldn't find 14 pros at the time we were ordering hardware. I 
have 14 nodes, each with a single 128 or 120GB SSD that serves as the boot 
drive  and the journal for 3 OSDs. And similarly, mine just started 
disappearing a few weeks ago. I've now had four fail (three 850 Pro, one 840 
Pro). I expect the rest to fail any day.

As it turns out I had a phone conversation with the support rep who has been 
helping me with RMA's today and he's putting together a report with my 
pertinent information in it to forward on to someone.

FWIW, I tried to get your 845's for this deploy, but couldn't find them 
anywhere, and since the 850's looked about as durable on paper I figured they 
would do ok. Seems not to be the case.

QH

On Fri, Sep 4, 2015 at 12:53 PM, Andrija Panic 
<andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>> wrote:
Hi James,

I had 3 CEPH nodes as folowing: 12 OSDs(HDD) and 2 SSDs (2x 6 Journals 
partitions on each SSD) - SSDs just vanished with no warning, no smartctl 
errors nothing... so 2 SSDs in 3 servers vanished in...2-3 weeks, after a 3-4 
months of being in production (VMs/KVM/CloudStack)

Mine were also Samsung 850 PRO 128GB.

Best,
Andrija

On 4 September 2015 at 19:27, James (Fei) Liu-SSI 
<james@ssi.samsung.com<mailto:james@ssi.samsung.com>> wrote:
Hi Quentin and Andrija,
Thanks so much for reporting the problems with Samsung.

Would be possible to get to know your configuration of your system?  What kind 
of workload are you running?  Do you use Samsung SSD as separate journaling 
disk, right?

Thanks so much.

James

From: ceph-users 
[mailto:ceph-users-boun...@lists.ceph.com<mailto:ceph-users-boun...@lists.ceph.com>]
 On Behalf Of Quentin Hartman
Sent: Thursday, September 03, 2015 1:06 PM
To: Andrija Panic
Cc: ceph-users
Subject: Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel 
s3700

Yeah, we've ordered some S3700's to replace them already. Should be here early 
next week. Hopefully they arrive before we have multiple nodes die at once and 
can no longer rebalance successfully.

Most of the drives I have are the 850 Pro 128GB (specifically MZ7KE128HMGA)
There are a couple 120GB 850 EVOs in there too, but ironically, none of them 
have pooped out yet.

On Thu, Sep 3, 2015 at 1:58 PM, Andrija Panic 
<andrija.pa...@gmail.com<mailto:andrija.pa...@gmail.com>> wrote:

I really advise removing the bastards becore they die...no rebalancing hapen

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

I just went through and ran this on all my currently running SSDs:

echo "$(smartctl -a /dev/sda | grep Total_LBAs_Written | awk '{ print $NF
}') * 512 /1025/1024/1024/1024" | bc

which is showing about 32TB written on the oldest nodes, about 20 on the
newer ones, and 1 on the first one I've RMA'd and replaced last week. So
the numbers are in-line with the test I did a few months ago in that they
are even, but looking back when I checked on them last my numbers were off
by 1024.

Note that this invocation of bc only outputs integers so the results will
be roudned.

On Fri, Sep 4, 2015 at 1:40 PM, James (Fei) Liu-SSI <
james@ssi.samsung.com> wrote:

> Hi Anrija,
>
> Your feedback is greatly appreciated.
>
>
>
> Regards,
>
> James
>
>
>
> *From:* Andrija Panic [mailto:andrija.pa...@gmail.com]
> *Sent:* Friday, September 04, 2015 12:39 PM
> *To:* James (Fei) Liu-SSI
> *Cc:* Quentin Hartman; ceph-users
>
> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T vs.
> Intel s3700
>
>
>
> James,
>
>
>
> there are simple FIO tests or even DD test on Linux, which you can run to
> see how good SSD will perform as CEPH Journal device (CEPH does writes with
> O_DIRECT and D_SYNC flags to SSDs) - Samsung 850 perform here extremely
> bad, as many, many other vendors (D_SYNC kills performance for them...)
>
>
>
> If you are not using D_SYNC flag, then Samsung can achieve some nice
> numbers...
>
> dd if=/dev/zero of=/dev/sda bs=4k count=10 oflag=direct,dsync (where
> /dev/sda is raw drive, or replace that with mount point i.e. /root/ddfile)
>
>
>
> Check post for more info please:
> http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>
> Thanks
>
>
>
> On 4 September 2015 at 21:31, James (Fei) Liu-SSI <
> james@ssi.samsung.com> wrote:
>
> Andrija,
>
> In your email thread, (18.000 (4Kb) IOPS constant write speed stands for
> 18K iops with 4k block size, right? However, you can only achieve 200IOPS
> with Samsung 850Pro, right?
>
>
>
> Theoretically, Samsung 850 Pro can get up to 100,000 IOPS with 4k Random
> Read with certain workload.  It is a little bit strange over here.
>
>
>
> Regards,
>
> James
>
>
>
>
>
> *From:* Andrija Panic [mailto:andrija.pa...@gmail.com]
> *Sent:* Friday, September 04, 2015 12:21 PM
> *To:* Quentin Hartman
> *Cc:* James (Fei) Liu-SSI; ceph-users
>
>
> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T vs.
> Intel s3700
>
>
>
> Quentin,
>
>
>
> try fio or dd with O_DIRECT and D_SYNC flags, and you will see less than
> 1MB/s - that is common for most "home" drives - check the post down to
> understand
>
> We removed all Samsung 850 pro 256GB from our new CEPH installation and
> replaced with Intel S3500 (18.000 (4Kb) IOPS constant write speed with
> O_DIRECT, D_SYNC, in comparison to 200 IOPS for Samsun 850pro - you can
> imagine the difference...):
>
>
> http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>
>
>
> Best
>
>
>
> On 4 September 2015 at 21:09, Quentin Hartman <
> qhart...@direwolfdigital.com> wrote:
>
> Mine are also mostly 850 Pros. I have a few 840s, and a few 850 EVOs in
> there just because I couldn't find 14 pros at the time we were ordering
> hardware. I have 14 nodes, each with a single 128 or 120GB SSD that serves
> as the boot drive  and the journal for 3 OSDs. And similarly, mine just
> started disappearing a few weeks ago. I've now had four fail (three 850
> Pro, one 840 Pro). I expect the rest to fail any day.
>
>
>
> As it turns out I had a phone conversation with the support rep who has
> been helping me with RMA's today and he's putting together a report with my
> pertinent information in it to forward on to someone.
>
>
>
> FWIW, I tried to get your 845's for this deploy, but couldn't find them
> anywhere, and since the 850's looked about as durable on paper I figured
> they would do ok. Seems not to be the case.
>
>
>
> QH
>
>
>
> On Fri, Sep 4, 2015 at 12:53 PM, Andrija Panic <andrija.pa...@gmail.com>
> wrote:
>
> Hi James,
>
>
>
> I had 3 CEPH nodes as folowing: 12 OSDs(HDD) and 2 SSDs (2x 6 Journals
> partitions on each SSD) - SSDs just vanished with no warning, no smartctl
> errors nothing... so 2 SSDs in 3 servers vanished in...2-3 weeks, after a
> 3-4 months of being in production (VMs/KVM/CloudStack)
>
> Mine were also Samsung 850 PRO 128GB.
>
>
>
> Best,
>
> Andrija
>
>
>
> On 4 September 2015 at 19

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

Hi James,

I had 3 CEPH nodes as folowing: 12 OSDs(HDD) and 2 SSDs (2x 6 Journals
partitions on each SSD) - SSDs just vanished with no warning, no smartctl
errors nothing... so 2 SSDs in 3 servers vanished in...2-3 weeks, after a
3-4 months of being in production (VMs/KVM/CloudStack)

Mine were also Samsung 850 PRO 128GB.

Best,
Andrija

On 4 September 2015 at 19:27, James (Fei) Liu-SSI <james@ssi.samsung.com
> wrote:

> Hi Quentin and Andrija,
>
> Thanks so much for reporting the problems with Samsung.
>
>
>
> Would be possible to get to know your configuration of your system?  What
> kind of workload are you running?  Do you use Samsung SSD as separate
> journaling disk, right?
>
>
>
> Thanks so much.
>
>
>
> James
>
>
>
> *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf
> Of *Quentin Hartman
> *Sent:* Thursday, September 03, 2015 1:06 PM
> *To:* Andrija Panic
> *Cc:* ceph-users
> *Subject:* Re: [ceph-users] which SSD / experiences with Samsung 843T vs.
> Intel s3700
>
>
>
> Yeah, we've ordered some S3700's to replace them already. Should be here
> early next week. Hopefully they arrive before we have multiple nodes die at
> once and can no longer rebalance successfully.
>
>
>
> Most of the drives I have are the 850 Pro 128GB (specifically
> MZ7KE128HMGA)
>
> There are a couple 120GB 850 EVOs in there too, but ironically, none of
> them have pooped out yet.
>
>
>
> On Thu, Sep 3, 2015 at 1:58 PM, Andrija Panic <andrija.pa...@gmail.com>
> wrote:
>
> I really advise removing the bastards becore they die...no rebalancing
> hapening just temp osd down while replacing journals...
>
> What size and model are yours Samsungs?
>
> On Sep 3, 2015 7:10 PM, "Quentin Hartman" <qhart...@direwolfdigital.com>
> wrote:
>
> We also just started having our 850 Pros die one after the other after
> about 9 months of service. 3 down, 11 to go... No warning at all, the drive
> is fine, and then it's not even visible to the machine. According to the
> stats in hdparm and the calcs I did they should have had years of life
> left, so it seems that ceph journals definitely do something they do not
> like, which is not reflected in their stats.
>
>
>
> QH
>
>
>
> On Wed, Aug 26, 2015 at 7:15 AM, 10 minus <t10te...@gmail.com> wrote:
>
> Hi ,
>
> We got a good deal on 843T and we are using it in our Openstack setup ..as
> journals .
> They have been running for last six months ... No issues .
>
> When we compared with  Intel SSDs I think it was 3700 they  were shade
> slower for our workload and considerably cheaper.
>
> We did not run any synthetic benchmark since we had a specific use case.
>
> The performance was better than our old setup so it was good enough.
>
> hth
>
>
>
> On Tue, Aug 25, 2015 at 12:07 PM, Andrija Panic <andrija.pa...@gmail.com>
> wrote:
>
> We have some 850 pro 256gb ssds if anyone interested to buy:)
>
> And also there was new 850 pro firmware that broke peoples disk which was
> revoked later etc... I'm sticking with only vacuum cleaners from Samsung
> for now, maybe... :)
>
> On Aug 25, 2015 12:02 PM, "Voloshanenko Igor" <igor.voloshane...@gmail.com>
> wrote:
>
> To be honest, Samsung 850 PRO not 24/7 series... it's something about
> desktop+ series, but anyway - results from this drives - very very bad in
> any scenario acceptable by real life...
>
>
>
> Possible 845 PRO more better, but we don't want to experiment anymore...
> So we choose S3500 240G. Yes, it's cheaper than S3700 (about 2x times), and
> no so durable for writes, but we think more better to replace 1 ssd per 1
> year than to pay double price now.
>
>
>
> 2015-08-25 12:59 GMT+03:00 Andrija Panic <andrija.pa...@gmail.com>:
>
> And should I mention that in another CEPH installation we had samsung 850
> pro 128GB and all of 6 ssds died in 2 month period - simply disappear from
> the system, so not wear out...
>
> Never again we buy Samsung :)
>
> On Aug 25, 2015 11:57 AM, "Andrija Panic" <andrija.pa...@gmail.com> wrote:
>
> First read please:
>
> http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>
> We are getting 200 IOPS in comparison to Intels3500 18.000 iops - those
> are  constant performance numbers, meaning avoiding drives cache and
> running for longer period of time...
> Also if checking with FIO you will get better latencies on intel s3500
> (model tested in our case) along with 20X better IOPS results...
>
> We observed original issue by having high speed at begining of i.e. file

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

We also just started having our 850 Pros die one after the other after
about 9 months of service. 3 down, 11 to go... No warning at all, the drive
is fine, and then it's not even visible to the machine. According to the
stats in hdparm and the calcs I did they should have had years of life
left, so it seems that ceph journals definitely do something they do not
like, which is not reflected in their stats.

QH

On Wed, Aug 26, 2015 at 7:15 AM, 10 minus  wrote:

> Hi ,
>
> We got a good deal on 843T and we are using it in our Openstack setup ..as
> journals .
> They have been running for last six months ... No issues .
> When we compared with  Intel SSDs I think it was 3700 they  were shade
> slower for our workload and considerably cheaper.
> We did not run any synthetic benchmark since we had a specific use case.
> The performance was better than our old setup so it was good enough.
>
> hth
>
>
>
> On Tue, Aug 25, 2015 at 12:07 PM, Andrija Panic 
> wrote:
>
>> We have some 850 pro 256gb ssds if anyone interested to buy:)
>>
>> And also there was new 850 pro firmware that broke peoples disk which was
>> revoked later etc... I'm sticking with only vacuum cleaners from Samsung
>> for now, maybe... :)
>> On Aug 25, 2015 12:02 PM, "Voloshanenko Igor" <
>> igor.voloshane...@gmail.com> wrote:
>>
>>> To be honest, Samsung 850 PRO not 24/7 series... it's something about
>>> desktop+ series, but anyway - results from this drives - very very bad in
>>> any scenario acceptable by real life...
>>>
>>> Possible 845 PRO more better, but we don't want to experiment anymore...
>>> So we choose S3500 240G. Yes, it's cheaper than S3700 (about 2x times), and
>>> no so durable for writes, but we think more better to replace 1 ssd per 1
>>> year than to pay double price now.
>>>
>>> 2015-08-25 12:59 GMT+03:00 Andrija Panic :
>>>
 And should I mention that in another CEPH installation we had samsung
 850 pro 128GB and all of 6 ssds died in 2 month period - simply disappear
 from the system, so not wear out...

 Never again we buy Samsung :)
 On Aug 25, 2015 11:57 AM, "Andrija Panic" 
 wrote:

> First read please:
>
> http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>
> We are getting 200 IOPS in comparison to Intels3500 18.000 iops -
> those are  constant performance numbers, meaning avoiding drives cache and
> running for longer period of time...
> Also if checking with FIO you will get better latencies on intel s3500
> (model tested in our case) along with 20X better IOPS results...
>
> We observed original issue by having high speed at begining of i.e.
> file transfer inside VM, which than halts to zero... We moved journals 
> back
> to HDDs and performans was acceptable...no we are upgrading to intel
> S3500...
>
> Best
> any details on that ?
>
> On Tue, 25 Aug 2015 11:42:47 +0200, Andrija Panic
>  wrote:
>
> > Make sure you test what ever you decide. We just learned this the
> hard way
> > with samsung 850 pro, which is total crap, more than you could
> imagine...
> >
> > Andrija
> > On Aug 25, 2015 11:25 AM, "Jan Schermer"  wrote:
> >
> > > I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
> > > Very cheap, better than Intel 3610 for sure (and I think it beats
> even
> > > 3700).
> > >
> > > Jan
> > >
> > > > On 25 Aug 2015, at 11:23, Christopher Kunz <
> chrisl...@de-punkt.de>
> > > wrote:
> > > >
> > > > Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
> > > >> Hi,
> > > >>
> > > >> most of the times I do get the recommendation from resellers to
> go with
> > > >> the intel s3700 for the journalling.
> > > >>
> > > > Check out the Intel s3610. 3 drive writes per day for 5 years.
> Plus, it
> > > > is cheaper than S3700.
> > > >
> > > > Regards,
> > > >
> > > > --ck
> > > > ___
> > > > ceph-users mailing list
> > > > ceph-users@lists.ceph.com
> > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> > >
> > > ___
> > > ceph-users mailing list
> > > ceph-users@lists.ceph.com
> > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> > >
>
>
>
> --
> Mariusz Gronczewski, Administrator
>
> Efigence S. A.
> ul. Wołoska 9a, 02-583 Warszawa
> T: [+48] 22 380 13 13
> F: [+48] 22 380 13 14
> E: mariusz.gronczew...@efigence.com
> 
>

 ___
 ceph-users mailing list
 

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Quentin Hartman]

Yeah, we've ordered some S3700's to replace them already. Should be here
early next week. Hopefully they arrive before we have multiple nodes die at
once and can no longer rebalance successfully.

Most of the drives I have are the 850 Pro 128GB (specifically MZ7KE128HMGA)
There are a couple 120GB 850 EVOs in there too, but ironically, none of
them have pooped out yet.

On Thu, Sep 3, 2015 at 1:58 PM, Andrija Panic 
wrote:

> I really advise removing the bastards becore they die...no rebalancing
> hapening just temp osd down while replacing journals...
>
> What size and model are yours Samsungs?
> On Sep 3, 2015 7:10 PM, "Quentin Hartman" 
> wrote:
>
>> We also just started having our 850 Pros die one after the other after
>> about 9 months of service. 3 down, 11 to go... No warning at all, the drive
>> is fine, and then it's not even visible to the machine. According to the
>> stats in hdparm and the calcs I did they should have had years of life
>> left, so it seems that ceph journals definitely do something they do not
>> like, which is not reflected in their stats.
>>
>> QH
>>
>> On Wed, Aug 26, 2015 at 7:15 AM, 10 minus  wrote:
>>
>>> Hi ,
>>>
>>> We got a good deal on 843T and we are using it in our Openstack setup
>>> ..as journals .
>>> They have been running for last six months ... No issues .
>>> When we compared with  Intel SSDs I think it was 3700 they  were shade
>>> slower for our workload and considerably cheaper.
>>> We did not run any synthetic benchmark since we had a specific use case.
>>> The performance was better than our old setup so it was good enough.
>>>
>>> hth
>>>
>>>
>>>
>>> On Tue, Aug 25, 2015 at 12:07 PM, Andrija Panic >> > wrote:
>>>
 We have some 850 pro 256gb ssds if anyone interested to buy:)

 And also there was new 850 pro firmware that broke peoples disk which
 was revoked later etc... I'm sticking with only vacuum cleaners from
 Samsung for now, maybe... :)
 On Aug 25, 2015 12:02 PM, "Voloshanenko Igor" <
 igor.voloshane...@gmail.com> wrote:

> To be honest, Samsung 850 PRO not 24/7 series... it's something about
> desktop+ series, but anyway - results from this drives - very very bad in
> any scenario acceptable by real life...
>
> Possible 845 PRO more better, but we don't want to experiment
> anymore... So we choose S3500 240G. Yes, it's cheaper than S3700 (about 2x
> times), and no so durable for writes, but we think more better to replace 
> 1
> ssd per 1 year than to pay double price now.
>
> 2015-08-25 12:59 GMT+03:00 Andrija Panic :
>
>> And should I mention that in another CEPH installation we had samsung
>> 850 pro 128GB and all of 6 ssds died in 2 month period - simply disappear
>> from the system, so not wear out...
>>
>> Never again we buy Samsung :)
>> On Aug 25, 2015 11:57 AM, "Andrija Panic" 
>> wrote:
>>
>>> First read please:
>>>
>>> http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>>>
>>> We are getting 200 IOPS in comparison to Intels3500 18.000 iops -
>>> those are  constant performance numbers, meaning avoiding drives cache 
>>> and
>>> running for longer period of time...
>>> Also if checking with FIO you will get better latencies on intel
>>> s3500 (model tested in our case) along with 20X better IOPS results...
>>>
>>> We observed original issue by having high speed at begining of i.e.
>>> file transfer inside VM, which than halts to zero... We moved journals 
>>> back
>>> to HDDs and performans was acceptable...no we are upgrading to intel
>>> S3500...
>>>
>>> Best
>>> any details on that ?
>>>
>>> On Tue, 25 Aug 2015 11:42:47 +0200, Andrija Panic
>>>  wrote:
>>>
>>> > Make sure you test what ever you decide. We just learned this the
>>> hard way
>>> > with samsung 850 pro, which is total crap, more than you could
>>> imagine...
>>> >
>>> > Andrija
>>> > On Aug 25, 2015 11:25 AM, "Jan Schermer"  wrote:
>>> >
>>> > > I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
>>> > > Very cheap, better than Intel 3610 for sure (and I think it
>>> beats even
>>> > > 3700).
>>> > >
>>> > > Jan
>>> > >
>>> > > > On 25 Aug 2015, at 11:23, Christopher Kunz <
>>> chrisl...@de-punkt.de>
>>> > > wrote:
>>> > > >
>>> > > > Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
>>> > > >> Hi,
>>> > > >>
>>> > > >> most of the times I do get the recommendation from resellers
>>> to go with
>>> > > >> the intel s3700 for the journalling.
>>> > > >>
>>> > > > Check out the Intel s3610. 3 drive 

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

I really advise removing the bastards becore they die...no rebalancing
hapening just temp osd down while replacing journals...

What size and model are yours Samsungs?
On Sep 3, 2015 7:10 PM, "Quentin Hartman" 
wrote:

> We also just started having our 850 Pros die one after the other after
> about 9 months of service. 3 down, 11 to go... No warning at all, the drive
> is fine, and then it's not even visible to the machine. According to the
> stats in hdparm and the calcs I did they should have had years of life
> left, so it seems that ceph journals definitely do something they do not
> like, which is not reflected in their stats.
>
> QH
>
> On Wed, Aug 26, 2015 at 7:15 AM, 10 minus  wrote:
>
>> Hi ,
>>
>> We got a good deal on 843T and we are using it in our Openstack setup
>> ..as journals .
>> They have been running for last six months ... No issues .
>> When we compared with  Intel SSDs I think it was 3700 they  were shade
>> slower for our workload and considerably cheaper.
>> We did not run any synthetic benchmark since we had a specific use case.
>> The performance was better than our old setup so it was good enough.
>>
>> hth
>>
>>
>>
>> On Tue, Aug 25, 2015 at 12:07 PM, Andrija Panic 
>> wrote:
>>
>>> We have some 850 pro 256gb ssds if anyone interested to buy:)
>>>
>>> And also there was new 850 pro firmware that broke peoples disk which
>>> was revoked later etc... I'm sticking with only vacuum cleaners from
>>> Samsung for now, maybe... :)
>>> On Aug 25, 2015 12:02 PM, "Voloshanenko Igor" <
>>> igor.voloshane...@gmail.com> wrote:
>>>
 To be honest, Samsung 850 PRO not 24/7 series... it's something about
 desktop+ series, but anyway - results from this drives - very very bad in
 any scenario acceptable by real life...

 Possible 845 PRO more better, but we don't want to experiment
 anymore... So we choose S3500 240G. Yes, it's cheaper than S3700 (about 2x
 times), and no so durable for writes, but we think more better to replace 1
 ssd per 1 year than to pay double price now.

 2015-08-25 12:59 GMT+03:00 Andrija Panic :

> And should I mention that in another CEPH installation we had samsung
> 850 pro 128GB and all of 6 ssds died in 2 month period - simply disappear
> from the system, so not wear out...
>
> Never again we buy Samsung :)
> On Aug 25, 2015 11:57 AM, "Andrija Panic" 
> wrote:
>
>> First read please:
>>
>> http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>>
>> We are getting 200 IOPS in comparison to Intels3500 18.000 iops -
>> those are  constant performance numbers, meaning avoiding drives cache 
>> and
>> running for longer period of time...
>> Also if checking with FIO you will get better latencies on intel
>> s3500 (model tested in our case) along with 20X better IOPS results...
>>
>> We observed original issue by having high speed at begining of i.e.
>> file transfer inside VM, which than halts to zero... We moved journals 
>> back
>> to HDDs and performans was acceptable...no we are upgrading to intel
>> S3500...
>>
>> Best
>> any details on that ?
>>
>> On Tue, 25 Aug 2015 11:42:47 +0200, Andrija Panic
>>  wrote:
>>
>> > Make sure you test what ever you decide. We just learned this the
>> hard way
>> > with samsung 850 pro, which is total crap, more than you could
>> imagine...
>> >
>> > Andrija
>> > On Aug 25, 2015 11:25 AM, "Jan Schermer"  wrote:
>> >
>> > > I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
>> > > Very cheap, better than Intel 3610 for sure (and I think it beats
>> even
>> > > 3700).
>> > >
>> > > Jan
>> > >
>> > > > On 25 Aug 2015, at 11:23, Christopher Kunz <
>> chrisl...@de-punkt.de>
>> > > wrote:
>> > > >
>> > > > Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
>> > > >> Hi,
>> > > >>
>> > > >> most of the times I do get the recommendation from resellers
>> to go with
>> > > >> the intel s3700 for the journalling.
>> > > >>
>> > > > Check out the Intel s3610. 3 drive writes per day for 5 years.
>> Plus, it
>> > > > is cheaper than S3700.
>> > > >
>> > > > Regards,
>> > > >
>> > > > --ck
>> > > > ___
>> > > > ceph-users mailing list
>> > > > ceph-users@lists.ceph.com
>> > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> > >
>> > > ___
>> > > ceph-users mailing list
>> > > ceph-users@lists.ceph.com
>> > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> > >
>>

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[10 minus]

Hi ,

We got a good deal on 843T and we are using it in our Openstack setup ..as
journals .
They have been running for last six months ... No issues .
When we compared with  Intel SSDs I think it was 3700 they  were shade
slower for our workload and considerably cheaper.
We did not run any synthetic benchmark since we had a specific use case.
The performance was better than our old setup so it was good enough.

hth



On Tue, Aug 25, 2015 at 12:07 PM, Andrija Panic andrija.pa...@gmail.com
wrote:

 We have some 850 pro 256gb ssds if anyone interested to buy:)

 And also there was new 850 pro firmware that broke peoples disk which was
 revoked later etc... I'm sticking with only vacuum cleaners from Samsung
 for now, maybe... :)
 On Aug 25, 2015 12:02 PM, Voloshanenko Igor igor.voloshane...@gmail.com
 wrote:

 To be honest, Samsung 850 PRO not 24/7 series... it's something about
 desktop+ series, but anyway - results from this drives - very very bad in
 any scenario acceptable by real life...

 Possible 845 PRO more better, but we don't want to experiment anymore...
 So we choose S3500 240G. Yes, it's cheaper than S3700 (about 2x times), and
 no so durable for writes, but we think more better to replace 1 ssd per 1
 year than to pay double price now.

 2015-08-25 12:59 GMT+03:00 Andrija Panic andrija.pa...@gmail.com:

 And should I mention that in another CEPH installation we had samsung
 850 pro 128GB and all of 6 ssds died in 2 month period - simply disappear
 from the system, so not wear out...

 Never again we buy Samsung :)
 On Aug 25, 2015 11:57 AM, Andrija Panic andrija.pa...@gmail.com
 wrote:

 First read please:

 http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

 We are getting 200 IOPS in comparison to Intels3500 18.000 iops - those
 are  constant performance numbers, meaning avoiding drives cache and
 running for longer period of time...
 Also if checking with FIO you will get better latencies on intel s3500
 (model tested in our case) along with 20X better IOPS results...

 We observed original issue by having high speed at begining of i.e.
 file transfer inside VM, which than halts to zero... We moved journals back
 to HDDs and performans was acceptable...no we are upgrading to intel
 S3500...

 Best
 any details on that ?

 On Tue, 25 Aug 2015 11:42:47 +0200, Andrija Panic
 andrija.pa...@gmail.com wrote:

  Make sure you test what ever you decide. We just learned this the
 hard way
  with samsung 850 pro, which is total crap, more than you could
 imagine...
 
  Andrija
  On Aug 25, 2015 11:25 AM, Jan Schermer j...@schermer.cz wrote:
 
   I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
   Very cheap, better than Intel 3610 for sure (and I think it beats
 even
   3700).
  
   Jan
  
On 25 Aug 2015, at 11:23, Christopher Kunz chrisl...@de-punkt.de
 
   wrote:
   
Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
Hi,
   
most of the times I do get the recommendation from resellers to
 go with
the intel s3700 for the journalling.
   
Check out the Intel s3610. 3 drive writes per day for 5 years.
 Plus, it
is cheaper than S3700.
   
Regards,
   
--ck
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
  
   ___
   ceph-users mailing list
   ceph-users@lists.ceph.com
   http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
  



 --
 Mariusz Gronczewski, Administrator

 Efigence S. A.
 ul. Wołoska 9a, 02-583 Warszawa
 T: [+48] 22 380 13 13
 F: [+48] 22 380 13 14
 E: mariusz.gronczew...@efigence.com
 mailto:mariusz.gronczew...@efigence.com


 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Christopher Kunz]

Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
 Hi,
 
 most of the times I do get the recommendation from resellers to go with
 the intel s3700 for the journalling.
 
Check out the Intel s3610. 3 drive writes per day for 5 years. Plus, it
is cheaper than S3700.

Regards,

--ck
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Mariusz Gronczewski]

any details on that ?

On Tue, 25 Aug 2015 11:42:47 +0200, Andrija Panic
andrija.pa...@gmail.com wrote:

 Make sure you test what ever you decide. We just learned this the hard way
 with samsung 850 pro, which is total crap, more than you could imagine...
 
 Andrija
 On Aug 25, 2015 11:25 AM, Jan Schermer j...@schermer.cz wrote:
 
  I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
  Very cheap, better than Intel 3610 for sure (and I think it beats even
  3700).
 
  Jan
 
   On 25 Aug 2015, at 11:23, Christopher Kunz chrisl...@de-punkt.de
  wrote:
  
   Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
   Hi,
  
   most of the times I do get the recommendation from resellers to go with
   the intel s3700 for the journalling.
  
   Check out the Intel s3610. 3 drive writes per day for 5 years. Plus, it
   is cheaper than S3700.
  
   Regards,
  
   --ck
   ___
   ceph-users mailing list
   ceph-users@lists.ceph.com
   http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 
  ___
  ceph-users mailing list
  ceph-users@lists.ceph.com
  http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 



-- 
Mariusz Gronczewski, Administrator

Efigence S. A.
ul. Wołoska 9a, 02-583 Warszawa
T: [+48] 22 380 13 13
F: [+48] 22 380 13 14
E: mariusz.gronczew...@efigence.com
mailto:mariusz.gronczew...@efigence.com


pgpoQ0v9f89v3.pgp
Description: OpenPGP digital signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

Make sure you test what ever you decide. We just learned this the hard way
with samsung 850 pro, which is total crap, more than you could imagine...

Andrija
On Aug 25, 2015 11:25 AM, Jan Schermer j...@schermer.cz wrote:

 I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
 Very cheap, better than Intel 3610 for sure (and I think it beats even
 3700).

 Jan

  On 25 Aug 2015, at 11:23, Christopher Kunz chrisl...@de-punkt.de
 wrote:
 
  Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
  Hi,
 
  most of the times I do get the recommendation from resellers to go with
  the intel s3700 for the journalling.
 
  Check out the Intel s3610. 3 drive writes per day for 5 years. Plus, it
  is cheaper than S3700.
 
  Regards,
 
  --ck
  ___
  ceph-users mailing list
  ceph-users@lists.ceph.com
  http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

First read please:
http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

We are getting 200 IOPS in comparison to Intels3500 18.000 iops - those
are  constant performance numbers, meaning avoiding drives cache and
running for longer period of time...
Also if checking with FIO you will get better latencies on intel s3500
(model tested in our case) along with 20X better IOPS results...

We observed original issue by having high speed at begining of i.e. file
transfer inside VM, which than halts to zero... We moved journals back to
HDDs and performans was acceptable...no we are upgrading to intel S3500...

Best
any details on that ?

On Tue, 25 Aug 2015 11:42:47 +0200, Andrija Panic
andrija.pa...@gmail.com wrote:

 Make sure you test what ever you decide. We just learned this the hard way
 with samsung 850 pro, which is total crap, more than you could imagine...

 Andrija
 On Aug 25, 2015 11:25 AM, Jan Schermer j...@schermer.cz wrote:

  I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
  Very cheap, better than Intel 3610 for sure (and I think it beats even
  3700).
 
  Jan
 
   On 25 Aug 2015, at 11:23, Christopher Kunz chrisl...@de-punkt.de
  wrote:
  
   Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
   Hi,
  
   most of the times I do get the recommendation from resellers to go
with
   the intel s3700 for the journalling.
  
   Check out the Intel s3610. 3 drive writes per day for 5 years. Plus,
it
   is cheaper than S3700.
  
   Regards,
  
   --ck
   ___
   ceph-users mailing list
   ceph-users@lists.ceph.com
   http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 
  ___
  ceph-users mailing list
  ceph-users@lists.ceph.com
  http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 



--
Mariusz Gronczewski, Administrator

Efigence S. A.
ul. Wołoska 9a, 02-583 Warszawa
T: [+48] 22 380 13 13
F: [+48] 22 380 13 14
E: mariusz.gronczew...@efigence.com
mailto:mariusz.gronczew...@efigence.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Voloshanenko Igor]

To be honest, Samsung 850 PRO not 24/7 series... it's something about
desktop+ series, but anyway - results from this drives - very very bad in
any scenario acceptable by real life...

Possible 845 PRO more better, but we don't want to experiment anymore... So
we choose S3500 240G. Yes, it's cheaper than S3700 (about 2x times), and no
so durable for writes, but we think more better to replace 1 ssd per 1 year
than to pay double price now.

2015-08-25 12:59 GMT+03:00 Andrija Panic andrija.pa...@gmail.com:

 And should I mention that in another CEPH installation we had samsung 850
 pro 128GB and all of 6 ssds died in 2 month period - simply disappear from
 the system, so not wear out...

 Never again we buy Samsung :)
 On Aug 25, 2015 11:57 AM, Andrija Panic andrija.pa...@gmail.com wrote:

 First read please:

 http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

 We are getting 200 IOPS in comparison to Intels3500 18.000 iops - those
 are  constant performance numbers, meaning avoiding drives cache and
 running for longer period of time...
 Also if checking with FIO you will get better latencies on intel s3500
 (model tested in our case) along with 20X better IOPS results...

 We observed original issue by having high speed at begining of i.e. file
 transfer inside VM, which than halts to zero... We moved journals back to
 HDDs and performans was acceptable...no we are upgrading to intel S3500...

 Best
 any details on that ?

 On Tue, 25 Aug 2015 11:42:47 +0200, Andrija Panic
 andrija.pa...@gmail.com wrote:

  Make sure you test what ever you decide. We just learned this the hard
 way
  with samsung 850 pro, which is total crap, more than you could
 imagine...
 
  Andrija
  On Aug 25, 2015 11:25 AM, Jan Schermer j...@schermer.cz wrote:
 
   I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
   Very cheap, better than Intel 3610 for sure (and I think it beats even
   3700).
  
   Jan
  
On 25 Aug 2015, at 11:23, Christopher Kunz chrisl...@de-punkt.de
   wrote:
   
Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
Hi,
   
most of the times I do get the recommendation from resellers to go
 with
the intel s3700 for the journalling.
   
Check out the Intel s3610. 3 drive writes per day for 5 years.
 Plus, it
is cheaper than S3700.
   
Regards,
   
--ck
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
  
   ___
   ceph-users mailing list
   ceph-users@lists.ceph.com
   http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
  



 --
 Mariusz Gronczewski, Administrator

 Efigence S. A.
 ul. Wołoska 9a, 02-583 Warszawa
 T: [+48] 22 380 13 13
 F: [+48] 22 380 13 14
 E: mariusz.gronczew...@efigence.com
 mailto:mariusz.gronczew...@efigence.com


 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

We have some 850 pro 256gb ssds if anyone interested to buy:)

And also there was new 850 pro firmware that broke peoples disk which was
revoked later etc... I'm sticking with only vacuum cleaners from Samsung
for now, maybe... :)
On Aug 25, 2015 12:02 PM, Voloshanenko Igor igor.voloshane...@gmail.com
wrote:

 To be honest, Samsung 850 PRO not 24/7 series... it's something about
 desktop+ series, but anyway - results from this drives - very very bad in
 any scenario acceptable by real life...

 Possible 845 PRO more better, but we don't want to experiment anymore...
 So we choose S3500 240G. Yes, it's cheaper than S3700 (about 2x times), and
 no so durable for writes, but we think more better to replace 1 ssd per 1
 year than to pay double price now.

 2015-08-25 12:59 GMT+03:00 Andrija Panic andrija.pa...@gmail.com:

 And should I mention that in another CEPH installation we had samsung 850
 pro 128GB and all of 6 ssds died in 2 month period - simply disappear from
 the system, so not wear out...

 Never again we buy Samsung :)
 On Aug 25, 2015 11:57 AM, Andrija Panic andrija.pa...@gmail.com
 wrote:

 First read please:

 http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

 We are getting 200 IOPS in comparison to Intels3500 18.000 iops - those
 are  constant performance numbers, meaning avoiding drives cache and
 running for longer period of time...
 Also if checking with FIO you will get better latencies on intel s3500
 (model tested in our case) along with 20X better IOPS results...

 We observed original issue by having high speed at begining of i.e. file
 transfer inside VM, which than halts to zero... We moved journals back to
 HDDs and performans was acceptable...no we are upgrading to intel S3500...

 Best
 any details on that ?

 On Tue, 25 Aug 2015 11:42:47 +0200, Andrija Panic
 andrija.pa...@gmail.com wrote:

  Make sure you test what ever you decide. We just learned this the hard
 way
  with samsung 850 pro, which is total crap, more than you could
 imagine...
 
  Andrija
  On Aug 25, 2015 11:25 AM, Jan Schermer j...@schermer.cz wrote:
 
   I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
   Very cheap, better than Intel 3610 for sure (and I think it beats
 even
   3700).
  
   Jan
  
On 25 Aug 2015, at 11:23, Christopher Kunz chrisl...@de-punkt.de
   wrote:
   
Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
Hi,
   
most of the times I do get the recommendation from resellers to
 go with
the intel s3700 for the journalling.
   
Check out the Intel s3610. 3 drive writes per day for 5 years.
 Plus, it
is cheaper than S3700.
   
Regards,
   
--ck
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
  
   ___
   ceph-users mailing list
   ceph-users@lists.ceph.com
   http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
  



 --
 Mariusz Gronczewski, Administrator

 Efigence S. A.
 ul. Wołoska 9a, 02-583 Warszawa
 T: [+48] 22 380 13 13
 F: [+48] 22 380 13 14
 E: mariusz.gronczew...@efigence.com
 mailto:mariusz.gronczew...@efigence.com


 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Götz Reinicke - IT Koordinator]

Hi,

most of the times I do get the recommendation from resellers to go with
the intel s3700 for the journalling.

Now I got an offer for systems with MLC 240 GB SATA Samsung 843T.

A quick research on google shows that that ssd is not as good as the
intel, but good, server grade 24/7 etc. and not that good in iops and
performance.

The samsung cost on the other hand nearly half in that offers ...

Dose anybody can give some feedback on the SSDs you use and in special
regarding the two I mention?

Thanks and regards. Götz




smime.p7s
Description: S/MIME Cryptographic Signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Jan Schermer]

I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
Very cheap, better than Intel 3610 for sure (and I think it beats even 3700).

Jan

 On 25 Aug 2015, at 11:23, Christopher Kunz chrisl...@de-punkt.de wrote:
 
 Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
 Hi,
 
 most of the times I do get the recommendation from resellers to go with
 the intel s3700 for the journalling.
 
 Check out the Intel s3610. 3 drive writes per day for 5 years. Plus, it
 is cheaper than S3700.
 
 Regards,
 
 --ck
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] which SSD / experiences with Samsung 843T vs. Intel s3700

[Andrija Panic]

And should I mention that in another CEPH installation we had samsung 850
pro 128GB and all of 6 ssds died in 2 month period - simply disappear from
the system, so not wear out...

Never again we buy Samsung :)
On Aug 25, 2015 11:57 AM, Andrija Panic andrija.pa...@gmail.com wrote:

 First read please:

 http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

 We are getting 200 IOPS in comparison to Intels3500 18.000 iops - those
 are  constant performance numbers, meaning avoiding drives cache and
 running for longer period of time...
 Also if checking with FIO you will get better latencies on intel s3500
 (model tested in our case) along with 20X better IOPS results...

 We observed original issue by having high speed at begining of i.e. file
 transfer inside VM, which than halts to zero... We moved journals back to
 HDDs and performans was acceptable...no we are upgrading to intel S3500...

 Best
 any details on that ?

 On Tue, 25 Aug 2015 11:42:47 +0200, Andrija Panic
 andrija.pa...@gmail.com wrote:

  Make sure you test what ever you decide. We just learned this the hard
 way
  with samsung 850 pro, which is total crap, more than you could imagine...
 
  Andrija
  On Aug 25, 2015 11:25 AM, Jan Schermer j...@schermer.cz wrote:
 
   I would recommend Samsung 845 DC PRO (not EVO, not just PRO).
   Very cheap, better than Intel 3610 for sure (and I think it beats even
   3700).
  
   Jan
  
On 25 Aug 2015, at 11:23, Christopher Kunz chrisl...@de-punkt.de
   wrote:
   
Am 25.08.15 um 11:18 schrieb Götz Reinicke - IT Koordinator:
Hi,
   
most of the times I do get the recommendation from resellers to go
 with
the intel s3700 for the journalling.
   
Check out the Intel s3610. 3 drive writes per day for 5 years. Plus,
 it
is cheaper than S3700.
   
Regards,
   
--ck
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
  
   ___
   ceph-users mailing list
   ceph-users@lists.ceph.com
   http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
  



 --
 Mariusz Gronczewski, Administrator

 Efigence S. A.
 ul. Wołoska 9a, 02-583 Warszawa
 T: [+48] 22 380 13 13
 F: [+48] 22 380 13 14
 E: mariusz.gronczew...@efigence.com
 mailto:mariusz.gronczew...@efigence.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com