Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-12 Thread Jasper Siero
. The problem is solved now :) Thanks! Jasper Van: Gregory Farnum [g...@gregs42.com] Verzonden: dinsdag 11 november 2014 19:19 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Tue, No

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-11 Thread Gregory Farnum
On Tue, Nov 11, 2014 at 5:06 AM, Jasper Siero wrote: > No problem thanks for helping. > I don't want to disable the deep scrubbing process itself because its very > useful but one placement group (3.30) is continuously deep scrubbing and it > should finish after some time but it won't. Hmm, how

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-11 Thread Jasper Siero
ory Farnum [g...@gregs42.com] Verzonden: maandag 10 november 2014 18:24 Aan: Jasper Siero CC: ceph-users; John Spray Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full It's supposed to do that; deep scrubbing is an ongoing consistency-check mechanism. If y

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-10 Thread Gregory Farnum
;>> >>>> http://www.mediafire.com/download/gikiy7cqs42cllt/ceph-mds.th1-mon001.log.tar.gz >>>> >>>> Thanks, >>>> >>>> Jasper >>>> >>>> >>>> Van: gr

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-10 Thread Jasper Siero
egards, Jasper Van: Gregory Farnum [g...@gregs42.com] Verzonden: vrijdag 7 november 2014 22:42 Aan: Jasper Siero CC: ceph-users; John Spray Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Thu, Nov 6, 2014 at 11:49 AM, Joh

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-10 Thread Jasper Siero
7 november 2014 22:42 Aan: Jasper Siero CC: ceph-users; John Spray Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Thu, Nov 6, 2014 at 11:49 AM, John Spray wrote: > This is still an issue on master, so a fix will be coming soon. > Follow the ti

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-07 Thread Gregory Farnum
e so I uploaded it to another one: >>> >>> http://www.mediafire.com/download/gikiy7cqs42cllt/ceph-mds.th1-mon001.log.tar.gz >>> >>> Thanks, >>> >>> Jasper >>> >>> >>> Van: gregory.far...@inktank.com [gregory.f

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-06 Thread John Spray
__ >> Van: gregory.far...@inktank.com [gregory.far...@inktank.com] namens Gregory >> Farnum [gfar...@redhat.com] >> Verzonden: donderdag 30 oktober 2014 1:03 >> Aan: Jasper Siero >> CC: John Spray; ceph-users >> Onderwerp: Re: [ceph-user

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-06 Thread John Spray
_ > Van: gregory.far...@inktank.com [gregory.far...@inktank.com] namens Gregory > Farnum [gfar...@redhat.com] > Verzonden: donderdag 30 oktober 2014 1:03 > Aan: Jasper Siero > CC: John Spray; ceph-users > Onderwerp: Re: [ceph-users] mds isn't working anymore after osd

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-03 Thread Jasper Siero
...@inktank.com [gregory.far...@inktank.com] namens Gregory Farnum [gfar...@redhat.com] Verzonden: donderdag 30 oktober 2014 1:03 Aan: Jasper Siero CC: John Spray; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Wed, Oct 29, 2014 at 7:51 AM, Jasper Si

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-30 Thread Jasper Siero
...@inktank.com [gregory.far...@inktank.com] namens Gregory Farnum [gfar...@redhat.com] Verzonden: donderdag 30 oktober 2014 1:03 Aan: Jasper Siero CC: John Spray; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Wed, Oct 29, 2014 at 7:51 AM, Jasper Si

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-29 Thread Gregory Farnum
On Wed, Oct 29, 2014 at 7:51 AM, Jasper Siero wrote: > Hello Greg, > > I added the debug options which you mentioned and started the process again: > > [root@th1-mon001 ~]# /usr/bin/ceph-mds -i th1-mon001 --pid-file > /var/run/ceph/mds.th1-mon001.pid -c /etc/ceph/ceph.conf --cluster ceph > --res

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-29 Thread Jasper Siero
...@inktank.com [gregory.far...@inktank.com] namens Gregory Farnum [gfar...@redhat.com] Verzonden: dinsdag 28 oktober 2014 19:26 Aan: Jasper Siero CC: John Spray; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full You'll need to gather a log with the

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-28 Thread Gregory Farnum
gt; >>>>> No problem thanks for looking into the log. I attached the log to this >>>>> email. >>>>> I'm looking forward for the new release because it would be nice to have >>>>> more possibilities to diagnose problems. >>>

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-24 Thread Jasper Siero
>> 1: /usr/bin/ceph-mds() [0x82ef61] >> 2: (()+0xf710) [0x7fec3d9a6710] >> 3: (gsignal()+0x35) [0x7fec3ca7c635] >> 4: (abort()+0x175) [0x7fec3ca7de15] >> 5: (__gnu_cxx::__verbose_terminate_handler()+0x12d) [0x7fec3d336a5d] >> 6: (()+0xbcbe6) [0x7fec3d334be6] >&g

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-16 Thread John Spray
) [0x7fec3d9a6710] >> 3: (gsignal()+0x35) [0x7fec3ca7c635] >> 4: (abort()+0x175) [0x7fec3ca7de15] >> 5: (__gnu_cxx::__verbose_terminate_handler()+0x12d) [0x7fec3d336a5d] >> 6: (()+0xbcbe6) [0x7fec3d334be6] >> 7: (()+0xbcc13) [0x7fec3d334c13] >> 8: (()+0x

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-16 Thread Jasper Siero
Farnum; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full Following up: firefly fix for undump is: https://github.com/ceph/ceph/pull/2734 Jasper: if you still need to try undumping on this existing firefly cluster, then you can download ceph-md

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-15 Thread John Spray
(ceph::__ceph_assert_fail(char const*, char const*, int, char > const*)+0x7f2) [0x94b812] > 10: /usr/bin/ceph-mds() [0x80f15e] > 11: (Dumper::undump(char const*)+0x65d) [0x56c7ad] > 12: (main()+0x1632) [0x569c62] > 13: (__libc_start_main()+0xfd) [0x7fec3ca68d5d] > 14: /usr/bin/ce

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-15 Thread Jasper Siero
mp -rdS ` is needed to interpret this. Aborted Jasper Van: Gregory Farnum [g...@inktank.com] Verzonden: dinsdag 14 oktober 2014 23:40 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full ceph-mds --undump-journal Looks

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-14 Thread Gregory Farnum
ot;undump" the journal back into ceph? > > Jasper > > > Van: Gregory Farnum [g...@inktank.com] > Verzonden: vrijdag 10 oktober 2014 23:45 > Aan: Jasper Siero > CC: ceph-users > Onderwerp: Re: [ceph-users] mds isn't working a

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-14 Thread Jasper Siero
o ceph? Jasper Van: Gregory Farnum [g...@inktank.com] Verzonden: vrijdag 10 oktober 2014 23:45 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full Ugh, "debug journaler", not "debug

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-10 Thread Gregory Farnum
> Van: Gregory Farnum [g...@inktank.com] > Verzonden: dinsdag 7 oktober 2014 19:45 > Aan: Jasper Siero > CC: ceph-users > Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full > > Sorry; I guess this fell o

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-07 Thread Gregory Farnum
gt; Jasper Siero > > > ________ > Van: Jasper Siero > Verzonden: donderdag 21 augustus 2014 16:43 > Aan: Gregory Farnum > Onderwerp: RE: [ceph-users] mds isn't working anymore after osd's running full > > I did restart it but you

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-08-20 Thread Gregory Farnum
2014 19:49 > Aan: Jasper Siero > CC: ceph-users@lists.ceph.com > Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full > > On Mon, Aug 18, 2014 at 6:56 AM, Jasper Siero > wrote: >> Hi all, >> >> We have a small ceph cluster running

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-08-20 Thread Jasper Siero
s, Jasper Van: Gregory Farnum [g...@inktank.com] Verzonden: dinsdag 19 augustus 2014 19:49 Aan: Jasper Siero CC: ceph-users@lists.ceph.com Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Mon, Aug 18, 2014 at 6:56 AM, Jasper Siero wrote: >

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-08-19 Thread Gregory Farnum
On Mon, Aug 18, 2014 at 6:56 AM, Jasper Siero wrote: > Hi all, > > We have a small ceph cluster running version 0.80.1 with cephfs on five > nodes. > Last week some osd's were full and shut itself down. To help de osd's start > again I added some extra osd's and moved some placement group director

[ceph-users] mds isn't working anymore after osd's running full

2014-08-18 Thread Jasper Siero
Hi all, We have a small ceph cluster running version 0.80.1 with cephfs on five nodes. Last week some osd's were full and shut itself down. To help de osd's start again I added some extra osd's and moved some placement group directories on the full osd's (which has a copy on another osd) to anot