Re: Ceph version 0.56.1, data loss on power failure

2013-01-10 Thread Wido den Hollander
On 01/10/2013 02:54 PM, Marcin Szukala wrote: Hi, I hava a ubuntu 12.10 host with image mapped with rbd module. The problem is that when this host face a power failure all data on the mapped image is lost. If I unmap the image via the rbd unmap command, the data is preserved. I've noticed this p

Re: Ceph version 0.56.1, data loss on power failure

2013-01-10 Thread Sylvain Munaut
Hi, > I hava a ubuntu 12.10 host with image mapped with rbd module. The > problem is that when this host > face a power failure all data on the mapped image is lost. If I unmap > the image via the rbd unmap command, the data is preserved. I've > noticed this problem also in version 0.56 Make sure

Re: Ceph version 0.56.1, data loss on power failure

2013-01-10 Thread Marcin Szukala
Hi, Scenario is correct but the last line. I can mount the image, but the data that was written to the image before power failure is lost. Currently the ceph cluster is not healthy, but i dont think its related because I had this issue before the cluster itsef had issues (about that I will write

Re: Ceph version 0.56.1, data loss on power failure

2013-01-10 Thread Gregory Farnum
On Thu, Jan 10, 2013 at 8:56 AM, Marcin Szukala wrote: > Hi, > > Scenario is correct but the last line. I can mount the image, but the > data that was written to the image before power failure is lost. > > Currently the ceph cluster is not healthy, but i dont think its > related because I had this

Re: Ceph version 0.56.1, data loss on power failure

2013-01-11 Thread Marcin Szukala
2013/1/10 Gregory Farnum : > On Thu, Jan 10, 2013 at 8:56 AM, Marcin Szukala > wrote: >> Hi, >> >> Scenario is correct but the last line. I can mount the image, but the >> data that was written to the image before power failure is lost. >> >> Currently the ceph cluster is not healthy, but i dont t

Re: Ceph version 0.56.1, data loss on power failure

2013-01-11 Thread Gregory Farnum
On Fri, Jan 11, 2013 at 3:07 AM, Marcin Szukala wrote: > 2013/1/10 Gregory Farnum : >> On Thu, Jan 10, 2013 at 8:56 AM, Marcin Szukala >> wrote: >>> Hi, >>> >>> Scenario is correct but the last line. I can mount the image, but the >>> data that was written to the image before power failure is los

Re: Ceph version 0.56.1, data loss on power failure

2013-01-11 Thread Marcin Szukala
Hi, The filesystem remains, but all of the data is lost. Regards, Marcin 2013/1/11 Gregory Farnum : > On Fri, Jan 11, 2013 at 3:07 AM, Marcin Szukala > wrote: >> 2013/1/10 Gregory Farnum : >>> On Thu, Jan 10, 2013 at 8:56 AM, Marcin Szukala >>> wrote: Hi, Scenario is correct but

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Marcin Szukala
Hi all, Any ideas how can I resolve my issue? Or where the problem is? Let me describe the issue. Host boots up and maps RBD image with XFS filesystems Host mounts the filesystems from the RBD image Host starts to write data to the mounted filesystems Host experiences power failure Host comes up

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Wido den Hollander
On 01/16/2013 11:50 AM, Marcin Szukala wrote: Hi all, Any ideas how can I resolve my issue? Or where the problem is? Let me describe the issue. Host boots up and maps RBD image with XFS filesystems Host mounts the filesystems from the RBD image Host starts to write data to the mounted filesys

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Dino Yancey
Hi Marcin, Not sure if anyone asked, but are your OSD journals on actual disk or are you using tmpfs? Dino On Wed, Jan 16, 2013 at 4:53 AM, Wido den Hollander wrote: > > > On 01/16/2013 11:50 AM, Marcin Szukala wrote: >> >> Hi all, >> >> Any ideas how can I resolve my issue? Or where the proble

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Yann Dupont
Le 16/01/2013 11:53, Wido den Hollander a écrit : On 01/16/2013 11:50 AM, Marcin Szukala wrote: Hi all, Any ideas how can I resolve my issue? Or where the problem is? Let me describe the issue. Host boots up and maps RBD image with XFS filesystems Host mounts the filesystems from the RBD i

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Marcin Szukala
Hi Dino, journals are on dedicated SSD Regard, Marcin 2013/1/16 Dino Yancey : > Hi Marcin, > > Not sure if anyone asked, but are your OSD journals on actual disk or > are you using tmpfs? > > Dino > > On Wed, Jan 16, 2013 at 4:53 AM, Wido den Hollander wrote: >> >> >> On 01/16/2013 11:50 AM, Ma

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Marcin Szukala
2013/1/16 Yann Dupont : > Le 16/01/2013 11:53, Wido den Hollander a écrit : > >> >> >> On 01/16/2013 11:50 AM, Marcin Szukala wrote: > > >>> Hi all, >>> >>> Any ideas how can I resolve my issue? Or where the problem is? >>> >>> Let me describe the issue. >>> Host boots up and maps RBD image with XF

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Jeff Mitchell
FWIW, my ceph data dirs (for e.g. mons) are all on XFS. I've experienced a lot of corruption on these on power loss to the node -- and in some cases even when power wasn't lost, and the box was simply rebooted. This is on Ubuntu 12.04 with the ceph-provied 3.6.3 kernel (as I'm using RBD on these).

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Sage Weil
On Wed, 16 Jan 2013, Wido den Hollander wrote: > > On 01/16/2013 11:50 AM, Marcin Szukala wrote: > > Hi all, > > > > Any ideas how can I resolve my issue? Or where the problem is? > > > > Let me describe the issue. > > Host boots up and maps RBD image with XFS filesystems > > Host mounts the fil

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Wido den Hollander
Op 16 jan. 2013 om 18:00 heeft Sage Weil het volgende geschreven: > On Wed, 16 Jan 2013, Wido den Hollander wrote: >> >> On 01/16/2013 11:50 AM, Marcin Szukala wrote: >>> Hi all, >>> >>> Any ideas how can I resolve my issue? Or where the problem is? >>> >>> Let me describe the issue. >>>

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Sage Weil
On Wed, 16 Jan 2013, Wido den Hollander wrote: > > Op 16 jan. 2013 om 18:00 heeft Sage Weil het volgende > geschreven: > > > On Wed, 16 Jan 2013, Wido den Hollander wrote: > >> > >> On 01/16/2013 11:50 AM, Marcin Szukala wrote: > >>> Hi all, > >>> > >>> Any ideas how can I resolve my issue? O

Re: Ceph version 0.56.1, data loss on power failure

2013-01-16 Thread Tomasz Paszkowski
Marcin is using kernel RBD. On Wed, Jan 16, 2013 at 7:10 PM, Sage Weil wrote: > On Wed, 16 Jan 2013, Wido den Hollander wrote: >> >> Op 16 jan. 2013 om 18:00 heeft Sage Weil het volgende >> geschreven: >> >> > On Wed, 16 Jan 2013, Wido den Hollander wrote: >> >> >> >> On 01/16/2013 11:50 AM, M

Re: Ceph version 0.56.1, data loss on power failure

2013-01-17 Thread Yann Dupont
Le 16/01/2013 17:56, Jeff Mitchell a écrit : FWIW, my ceph data dirs (for e.g. mons) are all on XFS. I've experienced a lot of corruption on these on power loss to the node -- and in some cases even when power wasn't lost, and the box was simply rebooted. This is on Ubuntu 12.04 with the ceph-pro