So, good, but price for 845 DC PRO 400 GB higher in about 2x times than intel S3500 240G (((
Any other models? ((( 2015-08-13 15:45 GMT+03:00 Jan Schermer <j...@schermer.cz>: > I tested and can recommend the Samsung 845 DC PRO (make sure it is DC PRO > and not just "PRO" or "DC EVO"!). > Those were very cheap but are out of stock at the moment (here). > Faster than Intels, cheaper, and slightly different technology (3D V-NAND) > which IMO makes them superior without needing many tricks to do its job. > > Jan > > On 13 Aug 2015, at 14:40, Voloshanenko Igor <igor.voloshane...@gmail.com> > wrote: > > Tnx, Irek! Will try! > > but another question to all, which SSD good enough for CEPH now? > > I'm looking into S3500 240G (I have some S3500 120G which show great > results. Around 8x times better than Samsung) > > Possible you can give advice about other vendors/models with same or below > price level as S3500 240G? > > 2015-08-13 12:11 GMT+03:00 Irek Fasikhov <malm...@gmail.com>: > >> Hi, Igor. >> Try to roll the patch here: >> >> http://www.theirek.com/blog/2014/02/16/patch-dlia-raboty-s-enierghoniezavisimym-keshiem-ssd-diskov >> >> P.S. I am no longer tracks changes in this direction(kernel), because we >> use already recommended SSD >> >> С уважением, Фасихов Ирек Нургаязович >> Моб.: +79229045757 >> >> 2015-08-13 11:56 GMT+03:00 Voloshanenko Igor <igor.voloshane...@gmail.com >> >: >> >>> So, after testing SSD (i wipe 1 SSD, and used it for tests) >>> >>> root@ix-s2:~# sudo fio --filename=/dev/sda --direct=1 --sync=1 >>> --rw=write --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based >>> --gr[53/1800] >>> ting --name=journal-test >>> journal-test: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, >>> iodepth=1 >>> fio-2.1.3 >>> Starting 1 process >>> Jobs: 1 (f=1): [W] [100.0% done] [0KB/1152KB/0KB /s] [0/288/0 iops] [eta >>> 00m:00s] >>> journal-test: (groupid=0, jobs=1): err= 0: pid=2849460: Thu Aug 13 >>> 10:46:42 2015 >>> write: io=68972KB, bw=1149.6KB/s, iops=287, runt= 60001msec >>> clat (msec): min=2, max=15, avg= 3.48, stdev= 1.08 >>> lat (msec): min=2, max=15, avg= 3.48, stdev= 1.08 >>> clat percentiles (usec): >>> | 1.00th=[ 2704], 5.00th=[ 2800], 10.00th=[ 2864], 20.00th=[ >>> 2928], >>> | 30.00th=[ 3024], 40.00th=[ 3088], 50.00th=[ 3280], 60.00th=[ >>> 3408], >>> | 70.00th=[ 3504], 80.00th=[ 3728], 90.00th=[ 3856], 95.00th=[ >>> 4016], >>> | 99.00th=[ 9024], 99.50th=[ 9280], 99.90th=[ 9792], >>> 99.95th=[10048], >>> | 99.99th=[14912] >>> bw (KB /s): min= 1064, max= 1213, per=100.00%, avg=1150.07, >>> stdev=34.31 >>> lat (msec) : 4=94.99%, 10=4.96%, 20=0.05% >>> cpu : usr=0.13%, sys=0.57%, ctx=17248, majf=0, minf=7 >>> IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >>> >=64=0.0% >>> submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >>> >=64=0.0% >>> complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >>> >=64=0.0% >>> issued : total=r=0/w=17243/d=0, short=r=0/w=0/d=0 >>> >>> Run status group 0 (all jobs): >>> WRITE: io=68972KB, aggrb=1149KB/s, minb=1149KB/s, maxb=1149KB/s, >>> mint=60001msec, maxt=60001msec >>> >>> Disk stats (read/write): >>> sda: ios=0/17224, merge=0/0, ticks=0/59584, in_queue=59576, util=99.30% >>> >>> So, it's pain... SSD do only 287 iops on 4K... 1,1 MB/s >>> >>> I try to change cache mode : >>> echo temporary write through > /sys/class/scsi_disk/2:0:0:0/cache_type >>> echo temporary write through > /sys/class/scsi_disk/3:0:0:0/cache_type >>> >>> no luck, still same shit results, also i found this article: >>> https://lkml.org/lkml/2013/11/20/264 pointed to old very simple patch, >>> which disable CMD_FLUSH >>> https://gist.github.com/TheCodeArtist/93dddcd6a21dc81414ba >>> >>> Has everybody better ideas, how to improve this? (or disable CMD_FLUSH >>> without recompile kernel, i used ubuntu and 4.0.4 for now (4.x branch >>> because SSD 850 Pro have issue with NCQ TRIM< and before 4.0.4 this >>> exception was not included into libsata.c) >>> >>> 2015-08-12 19:17 GMT+03:00 Pieter Koorts <pieter.koo...@me.com>: >>> >>>> Hi Igor >>>> >>>> I suspect you have very much the same problem as me. >>>> >>>> https://www.mail-archive.com/ceph-users@lists.ceph.com/msg22260.html >>>> >>>> Basically Samsung drives (like many SATA SSD's) are very much hit and >>>> miss so you will need to test them like described here to see if they are >>>> any good. >>>> http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/ >>>> >>>> To give you an idea my average performance went from 11MB/s (with >>>> Samsung SSD) to 30MB/s (without any SSD) on write performance. This is a >>>> very small cluster. >>>> >>>> Pieter >>>> >>>> On Aug 12, 2015, at 04:33 PM, Voloshanenko Igor < >>>> igor.voloshane...@gmail.com> wrote: >>>> >>>> Hi all, we have setup CEPH cluster with 60 OSD (2 diff types) (5 nodes, >>>> 12 disks on each, 10 HDD, 2 SSD) >>>> >>>> Also we cover this with custom crushmap with 2 root leaf >>>> >>>> ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY >>>> -100 5.00000 root ssd >>>> -102 1.00000 host ix-s2-ssd >>>> 2 1.00000 osd.2 up 1.00000 1.00000 >>>> 9 1.00000 osd.9 up 1.00000 1.00000 >>>> -103 1.00000 host ix-s3-ssd >>>> 3 1.00000 osd.3 up 1.00000 1.00000 >>>> 7 1.00000 osd.7 up 1.00000 1.00000 >>>> -104 1.00000 host ix-s5-ssd >>>> 1 1.00000 osd.1 up 1.00000 1.00000 >>>> 6 1.00000 osd.6 up 1.00000 1.00000 >>>> -105 1.00000 host ix-s6-ssd >>>> 4 1.00000 osd.4 up 1.00000 1.00000 >>>> 8 1.00000 osd.8 up 1.00000 1.00000 >>>> -106 1.00000 host ix-s7-ssd >>>> 0 1.00000 osd.0 up 1.00000 1.00000 >>>> 5 1.00000 osd.5 up 1.00000 1.00000 >>>> -1 5.00000 root platter >>>> -2 1.00000 host ix-s2-platter >>>> 13 1.00000 osd.13 up 1.00000 1.00000 >>>> 17 1.00000 osd.17 up 1.00000 1.00000 >>>> 21 1.00000 osd.21 up 1.00000 1.00000 >>>> 27 1.00000 osd.27 up 1.00000 1.00000 >>>> 32 1.00000 osd.32 up 1.00000 1.00000 >>>> 37 1.00000 osd.37 up 1.00000 1.00000 >>>> 44 1.00000 osd.44 up 1.00000 1.00000 >>>> 48 1.00000 osd.48 up 1.00000 1.00000 >>>> 55 1.00000 osd.55 up 1.00000 1.00000 >>>> 59 1.00000 osd.59 up 1.00000 1.00000 >>>> -3 1.00000 host ix-s3-platter >>>> 14 1.00000 osd.14 up 1.00000 1.00000 >>>> 18 1.00000 osd.18 up 1.00000 1.00000 >>>> 23 1.00000 osd.23 up 1.00000 1.00000 >>>> 28 1.00000 osd.28 up 1.00000 1.00000 >>>> 33 1.00000 osd.33 up 1.00000 1.00000 >>>> 39 1.00000 osd.39 up 1.00000 1.00000 >>>> 43 1.00000 osd.43 up 1.00000 1.00000 >>>> 47 1.00000 osd.47 up 1.00000 1.00000 >>>> 54 1.00000 osd.54 up 1.00000 1.00000 >>>> 58 1.00000 osd.58 up 1.00000 1.00000 >>>> -4 1.00000 host ix-s5-platter >>>> 11 1.00000 osd.11 up 1.00000 1.00000 >>>> 16 1.00000 osd.16 up 1.00000 1.00000 >>>> 22 1.00000 osd.22 up 1.00000 1.00000 >>>> 26 1.00000 osd.26 up 1.00000 1.00000 >>>> 31 1.00000 osd.31 up 1.00000 1.00000 >>>> 36 1.00000 osd.36 up 1.00000 1.00000 >>>> 41 1.00000 osd.41 up 1.00000 1.00000 >>>> 46 1.00000 osd.46 up 1.00000 1.00000 >>>> 51 1.00000 osd.51 up 1.00000 1.00000 >>>> 56 1.00000 osd.56 up 1.00000 1.00000 >>>> -5 1.00000 host ix-s6-platter >>>> 12 1.00000 osd.12 up 1.00000 1.00000 >>>> 19 1.00000 osd.19 up 1.00000 1.00000 >>>> 24 1.00000 osd.24 up 1.00000 1.00000 >>>> 29 1.00000 osd.29 up 1.00000 1.00000 >>>> 34 1.00000 osd.34 up 1.00000 1.00000 >>>> 38 1.00000 osd.38 up 1.00000 1.00000 >>>> 42 1.00000 osd.42 up 1.00000 1.00000 >>>> 50 1.00000 osd.50 up 1.00000 1.00000 >>>> 53 1.00000 osd.53 up 1.00000 1.00000 >>>> 57 1.00000 osd.57 up 1.00000 1.00000 >>>> -6 1.00000 host ix-s7-platter >>>> 10 1.00000 osd.10 up 1.00000 1.00000 >>>> 15 1.00000 osd.15 up 1.00000 1.00000 >>>> 20 1.00000 osd.20 up 1.00000 1.00000 >>>> 25 1.00000 osd.25 up 1.00000 1.00000 >>>> 30 1.00000 osd.30 up 1.00000 1.00000 >>>> 35 1.00000 osd.35 up 1.00000 1.00000 >>>> 40 1.00000 osd.40 up 1.00000 1.00000 >>>> 45 1.00000 osd.45 up 1.00000 1.00000 >>>> 49 1.00000 osd.49 up 1.00000 1.00000 >>>> 52 1.00000 osd.52 up 1.00000 1.00000 >>>> >>>> >>>> Then create 2 pools, 1 on HDD (platters), 1 on SSD/ >>>> and put SSD pul in from of HDD pool (cache tier) >>>> >>>> now we receive very bad performance results from cluster. >>>> Even with rados bench we received very unstable performance with even >>>> zero speed. So it's create very big issues for our clients. >>>> >>>> I try to tune all possible values, including OSD, but still no luck. >>>> >>>> Also very unbelievble situation, when i do >>>> ceph tell... bench on SSD OSD - i receive about 20MB/s >>>> If for HDD - 67 MB/s... >>>> >>>> I don;t understand why cache pools which consist of SSD works so bad... >>>> We used Samsung 850 Pro 256 Gb as SSDs >>>> >>>> Can you guys give me advice please... >>>> >>>> Also very idiotic thing, when i set cache-mode to forward and try to >>>> flush-evict all object (not all object evicted, some busy (locked on KVM >>>> sides). but now i receive quite stable results for rados bench >>>> >>>> Total time run: 30.275871 >>>> Total writes made: 2076 >>>> Write size: 4194304 >>>> Bandwidth (MB/sec): 274.278 >>>> >>>> Stddev Bandwidth: 75.1445 >>>> Max bandwidth (MB/sec): 368 >>>> Min bandwidth (MB/sec): 0 >>>> Average Latency: 0.232892 >>>> Stddev Latency: 0.240356 >>>> Max latency: 2.01436 >>>> Min latency: 0.0716344 >>>> >>>> Without zeros, etc... So i don't understand how it's possible. >>>> >>>> Also interesting thing, when i disable overlay for pool, rados bench >>>> become around 70MB/s as for ordinary HDD, but in same time rados bench for >>>> SSD pool, which not used anymore show same bad results... >>>> >>>> So please, give me some direction to deeg... >>>> >>>> >>>> _______________________________________________ >>>> ceph-users mailing list >>>> ceph-users@lists.ceph.com >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>> >>>> >>> >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@lists.ceph.com >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >>> >> > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
