Re: [ceph-users] xfs/nobarrier

2014-12-29 Thread Ric Wheeler
On 12/27/2014 02:32 AM, Lindsay Mathieson wrote: I see a lot of people mount their xfs osd's with nobarrier for extra performance, certainly it makes a huge difference to my small system. However I don't do it as my understanding is this runs a risk of data corruption in the event of power failu

Re: [ceph-users] xfs/nobarrier

2014-12-29 Thread Lindsay Mathieson
On Mon, 29 Dec 2014 07:04:47 PM Mark Kirkwood wrote: > > Thanks all, I'll definitely stick with nobarrier > > Maybe you meant to say *barrier* ? Oops :) Yah -- Lindsay signature.asc Description: This is a digitally signed message part. ___ ceph-users

Re: [ceph-users] xfs/nobarrier

2014-12-28 Thread Mark Kirkwood
On 29/12/14 02:46, Lindsay Mathieson wrote: On Sat, 27 Dec 2014 09:41:19 PM you wrote: I certainly wouldn't, I've seen utility power fail and the transfer switch fail to transition to UPS strings. Had this happened to me with nobarrier it would have been a very sad day. I'd second that. In

Re: [ceph-users] xfs/nobarrier

2014-12-28 Thread Lindsay Mathieson
On Sat, 27 Dec 2014 09:41:19 PM you wrote: > > > I certainly wouldn't, I've seen utility power fail and the transfer > > > switch fail to transition to UPS strings. Had this happened to me with > > > nobarrier it would have been a very sad day. > > > > > > > > > > > > > > I'd second that. In addi

Re: [ceph-users] xfs/nobarrier

2014-12-27 Thread Sage Weil
On Sun, 28 Dec 2014, Mark Kirkwood wrote: > On 28/12/14 15:51, Kyle Bader wrote: > > > do people consider a UPS + Shutdown procedures a suitable substitute? > > > > I certainly wouldn't, I've seen utility power fail and the transfer > > switch fail to transition to UPS strings. Had this happened t

Re: [ceph-users] xfs/nobarrier

2014-12-27 Thread Mark Kirkwood
On 28/12/14 15:51, Kyle Bader wrote: do people consider a UPS + Shutdown procedures a suitable substitute? I certainly wouldn't, I've seen utility power fail and the transfer switch fail to transition to UPS strings. Had this happened to me with nobarrier it would have been a very sad day. I

Re: [ceph-users] xfs/nobarrier

2014-12-27 Thread Kyle Bader
> do people consider a UPS + Shutdown procedures a suitable substitute? I certainly wouldn't, I've seen utility power fail and the transfer switch fail to transition to UPS strings. Had this happened to me with nobarrier it would have been a very sad day. -- Kyle Bader _

Re: [ceph-users] xfs/nobarrier

2014-12-27 Thread Andrey Korolyov
On Sun, Dec 28, 2014 at 1:25 AM, Lindsay Mathieson wrote: > On Sat, 27 Dec 2014 06:02:32 PM you wrote: >> Are you able to separate log with data in your setup and check the >> difference? > > Do you mean putting the OSD journal on a separate disk? I have the journals on > SSD partitions, which has

Re: [ceph-users] xfs/nobarrier

2014-12-27 Thread Lindsay Mathieson
On Sat, 27 Dec 2014 06:02:32 PM you wrote: > Are you able to separate log with data in your setup and check the > difference? Do you mean putting the OSD journal on a separate disk? I have the journals on SSD partitions, which has helped a lot, previously I was getting 13 MB/s Its not a good SS

Re: [ceph-users] xfs/nobarrier

2014-12-27 Thread Andrey Korolyov
On Sat, Dec 27, 2014 at 4:31 PM, Lindsay Mathieson wrote: > On Sat, 27 Dec 2014 04:59:51 PM you wrote: >> Power supply means bigger capex and less redundancy, as the emergency >> procedure in case of power failure is less deterministic than with >> controlled battery-backed cache. > > Yes, the who

Re: [ceph-users] xfs/nobarrier

2014-12-27 Thread Lindsay Mathieson
On Sat, 27 Dec 2014 04:59:51 PM you wrote: > Power supply means bigger capex and less redundancy, as the emergency > procedure in case of power failure is less deterministic than with > controlled battery-backed cache. Yes, the whole auto shut-down procedure is rather more complex and fragile f

Re: [ceph-users] xfs/nobarrier

2014-12-27 Thread Andrey Korolyov
Power supply means bigger capex and less redundancy, as the emergency procedure in case of power failure is less deterministic than with controlled battery-backed cache. Cache battery is smaller and way more predictable for a health measurements than a UPS (if passes internal check, it will be *alw

Re: [ceph-users] xfs/nobarrier

2014-12-27 Thread Lindsay Mathieson
On Sat, 27 Dec 2014 09:03:16 PM Mark Kirkwood wrote: > Yep. If you have 'em plugged into a RAID/HBA card with a battery backup > (that also disables their individual caches) then it is safe to use > nobarrier, otherwise data corruption will result if the server > experiences power loss. Thanks

Re: [ceph-users] xfs/nobarrier

2014-12-27 Thread Mark Kirkwood
On 27/12/14 20:32, Lindsay Mathieson wrote: I see a lot of people mount their xfs osd's with nobarrier for extra performance, certainly it makes a huge difference to my small system. However I don't do it as my understanding is this runs a risk of data corruption in the event of power failure -

[ceph-users] xfs/nobarrier

2014-12-26 Thread Lindsay Mathieson
I see a lot of people mount their xfs osd's with nobarrier for extra performance, certainly it makes a huge difference to my small system. However I don't do it as my understanding is this runs a risk of data corruption in the event of power failure - this is the case, even with ceph? side not