Re: Can't start ceph mon

2012-11-26 Thread Gregory Farnum
On Fri, Nov 23, 2012 at 12:25 AM, Dave (Bob) d...@bob-the-boat.me.uk wrote: Greg, Is there any news on this? David Hurray, there is! Your latest stashed version of the PGMap got corrupted; knowing that helped me to track down a bug — we weren't checking the return value of an fsync! Fixes to

Re: Can't start ceph mon

2012-11-20 Thread Dave (Bob)
Do you have other monitors in working order? The easiest way to handle it if that's the case is just to remove this monitor from the cluster and add it back in as a new monitor with a fresh store. If not we can look into reconstructing it. -Greg Also, if you still have it, could you zip up your

Can't start ceph mon

2012-11-19 Thread Dave Humphreys (Datatone)
I have a problem in which I can't start my ceph monitor. The log is shown below. The log shows version 0.54. I was running 0.52 when the problem arose, and I moved to the latest in case the newer version fixed the problem. The original failure happened a week or so ago, and could have been as

Can't Start Ceph Mon

2012-11-19 Thread Dave Humphreys (Bob)
I can't start my ceph monitor, the log is attached below. Whilst the log shows 0.54, the problem arose with 0.52, and may have been caused when disk space ran out as a result of a huge set of ceph log files. Is there a way to recover? Ragards, David bash-4.1# cat

Re: Can't start ceph mon

2012-11-19 Thread Gregory Farnum
On Mon, Nov 19, 2012 at 1:08 PM, Dave Humphreys (Datatone) d...@datatone.co.uk wrote: I have a problem in which I can't start my ceph monitor. The log is shown below. The log shows version 0.54. I was running 0.52 when the problem arose, and I moved to the latest in case the newer version

Re: Can't start ceph mon

2012-11-19 Thread Gregory Farnum
Also, if you still have it, could you zip up your monitor data directory and put it somewhere accessible to us? (I can provide you a drop point if necessary.) We'd like to look at the file layouts a bit since we thought we were properly handling ENOSPC-style issues. -Greg On Mon, Nov 19, 2012 at