- hope this is as
expected. Our devel system cannot send mail directly.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Werner-Voß-Damm 62 Fax: +49 30 99296856
12101 Berlin http://www.m-privacy.de
Amtsgericht Charlottenburg, HRB 84946
Geschäftsführer:
Dipl
in the .conf file
mon_host line by hand after new and before mon create-initial.
With that change, all commands work as expected.
Thanks,
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Werner-Voß-Damm 62 Fax: +49 30 99296856
12101 Berlin http://www.m-privacy.de
Hello Ceph!
The Ceph init script (src/init-ceph.in) creates pid files without
cluster names. This means that only one cluster can run at a time. The
solution is simple and works fine here, patch against 0.94 is attached.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am 14.10.2014 16:23, schrieb Sage Weil:
On Tue, 14 Oct 2014, Amon Ott wrote:
Am 13.10.2014 20:16, schrieb Sage Weil:
We've been doing a lot of work on CephFS over the past few months. This
is an update on the current state of things as of Giant.
...
* Either the kernel client (kernel 3.17
Am 15.10.2014 14:11, schrieb Ric Wheeler:
On 10/15/2014 08:43 AM, Amon Ott wrote:
Am 14.10.2014 16:23, schrieb Sage Weil:
On Tue, 14 Oct 2014, Amon Ott wrote:
Am 13.10.2014 20:16, schrieb Sage Weil:
We've been doing a lot of work on CephFS over the past few months.
This
is an update
of severe known
problems we want to avoid Fuse. How good are our chances of a stable
system with the kernel client in the latest longterm kernel 3.14? Will
there be further bugfixes or feature backports?
Thanks again,
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Werner-Voß
tag in the pull to
local tree by some reason. Repository cloning from scratch resolved
this :)
This is normal git behaviour. git fetch -t is your friend.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 99296856
10179 Berlin
and also
fixed kernel 3.8 code. We have been waiting to replace our existing
cluster fs with CephFS for at least one and a half years, but it has
never been stable enough in our setup with standby MDS and takeover, let
alone multi-active. We do not want to risk a complete cluster failure.
Amon Ott
, some load sharing for MDS would be nice to
have. I would be willing to help with tests and reports, if needed.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 99296856
10179 Berlin http://www.m-privacy.de
Amtsgericht
Am 05.11.2012 21:17, schrieb Cláudio Martins:
On Fri, 1 Jun 2012 11:35:37 +0200 Amon Ott a@m-privacy.de wrote:
After backporting syncfs() support into Debian stable libc6 2.11 and
recompiling Ceph with it, our test cluster is now running with syncfs().
Hi,
We're running OSDs
and performance is being worked on all the time.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 99296856
10179 Berlin http://www.m-privacy.de
Amtsgericht Charlottenburg, HRB 84946
Geschäftsführer:
Dipl.-Kfm. Holger Maczkowsky
On Tuesday 03 July 2012 wrote Sage Weil:
On Tue, 3 Jul 2012, Amon Ott wrote:
just found out that some Debian binary packages do not get stripped - a
53MB ceph-mds does look a bit weird.
Identified packages ceph-mds and gceph and added these lines:
dh_strip -pceph-mds --dbg
before going on holidays there might still
be a chance.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 24342336
10179 Berlin http://www.m-privacy.de
Amtsgericht Charlottenburg, HRB 84946
Geschäftsführer:
Dipl.-Kfm. Holger
Hi folks,
just found out that some Debian binary packages do not get stripped - a 53MB
ceph-mds does look a bit weird.
Identified packages ceph-mds and gceph and added these lines:
dh_strip -pceph-mds --dbg-package=ceph-mds-dbg
dh_strip -pgceph --dbg-package=gceph-dbg
Amon Ott
to work.
Today's MDS log is available at
https://download.m-privacy.de/homeuser-mds.0.log.gz
Is this a known problem? It has been with us for a looong time now, but since
rebooting used to help, we never tracked it down.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am
On Wednesday 06 June 2012 wrote Stefan Priebe - Profihost AG:
Am 06.06.2012 12:57, schrieb Amon Ott:
On Wednesday 06 June 2012 wrote Stefan Priebe - Profihost AG:
Hi Amon,
i've added your patch:
# strings /lib/libc-2.11.3.so |grep -i syncfs
syncfs
But configure of ceph still
built 2.11.3-3 source tree. It
needs to be last in the series file, because several existing Debian patches
modify the sources at various places. I could also make our compiled packages
available to you for download.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am
libc6) with dpkg-buildpackage, install new
libc6-dev, rebuild ceph packages against it, install and retry. AFAIK, not
even libc6 in Debian experimental has syncfs() support.
Also see thread OSD deadlock with cephfs client and OSD on same machine
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH
On Wednesday 30 May 2012 wrote Amon Ott:
On Tuesday 29 May 2012 you wrote:
On Tue, 29 May 2012, Amon Ott wrote:
Please consider putting out a fat warning at least at build time, if
syncfs() is not available, e.g. No syncfs() syscall, please expect a
deadlock when running osd on non
On Tuesday 29 May 2012 you wrote:
On Tue, 29 May 2012, Amon Ott wrote:
Conclusion: If you want to run OSD and cephfs kernel client on the same
Linux server and have a libc6 before 2.14 (e.g. Debian's newest in
experimental is 2.13) or a kernel before 2.6.39, either do not use ext4
tried to recover, all MDS
instances died with those symptoms. It seems that a partial sync of journal
or data partition causes that broken state.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 24342336
10179 Berlin http
On Wednesday 23 May 2012 wrote Tommi Virtanen:
On Wed, May 23, 2012 at 2:00 AM, Amon Ott a@m-privacy.de wrote:
So I started experimenting with the new cluster variable, but it does
not seem to be well supported so far. mkcephfs does not even know about
it and always uses ceph as cluster
On Thursday 24 May 2012 wrote Amon Ott:
Attached is a patch based on current git stable that makes mkcephfs work
fine for me with --cluster name. ceph-mon uses the wrong mkfs path for mon
data (default ceph instead of supplied cluster name), so I put in a
workaround.
Please have a look
cluster names?
3) Are there plans to support cluster names in mkcephfs?
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 24342336
10179 Berlin http://www.m-privacy.de
Amtsgericht Charlottenburg, HRB 84946
Geschäftsführer
tried -n.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 24342336
10179 Berlin http://www.m-privacy.de
Amtsgericht Charlottenburg, HRB 84946
Geschäftsführer:
Dipl.-Kfm. Holger Maczkowsky,
Roman Maczkowsky
GnuPG-Key-ID
On Thursday 02 February 2012 wrote Gregory Farnum:
On Wed, Feb 1, 2012 at 9:02 AM, Amon Ott a@m-privacy.de wrote:
ceph should have recovered here. Might also be caused by this setting
that I tried for a while, it is off now:
mds standby replay = true
With this setting, if the active
On Tuesday 31 January 2012 wrote Gregory Farnum:
On Tue, Jan 31, 2012 at 4:00 AM, Amon Ott a@m-privacy.de wrote:
Hi again!
We are running Ceph 0.41 and kernel 3.2.2 with current for-linus code
(commit 3d882ce47de80e0294a536bec771b5651885b4d3) now.
After some heavy workloads we see
or a new bug? Can it be related to .snap pseudo dirs? The
problem appeared without ever using snapshots, though.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 24342336
10179 Berlin http://www.m-privacy.de
Amtsgericht
On Thursday 29 December 2011 wrote Amon Ott:
I finally got the test cluster freed up for more Ceph testing.
On Friday 23 December 2011 wrote Gregory Farnum:
Unfortunately there's not enough info in this log either. If you can
reproduce it with mds debug = 20 and put that log somewhere
Hi folks,
the current code in 32 Bit use 1 as fallback number, if the inode number
calculation returns 0. This always collides with the inode number of the root
inode. The attached patch changes the fallback number to 2, which seems to be
unused on our test systems.
Amon Ott
--
Dr. Amon Ott
going on, though. Sorry. :(
-Greg
Here is what MDS logs with debug 20. No idea if it really helps. The cluster
is still in the broken state, should I try to reproduce with a recreated ceph
fs and debug 20? This could be GBs of logs.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30
On Friday 23 December 2011 wrote Gregory Farnum:
Unfortunately there's not enough info in this log either. If you can
reproduce it with mds debug = 20 and put that log somewhere, it
ought to be enough to work out what's going on, though. Sorry. :(
Will do that as soon as I can.
Amon Ott
On Thursday 15 December 2011 wrote Gregory Farnum:
On Thu, Dec 8, 2011 at 5:55 AM, Amon Ott a@m-privacy.de wrote:
Hi folks,
if file access through Ceph kernel client cannot continue, e.g. because
there is no mds available, it hangs forever.
I would prefer if after a timeout
Sorry folks for making so much trouble,
but there is still another locking issue with the kernel client. iput gets
called with locks held, but iput can sleep and did in this occasion. It did
not hang the system this time, but it can be quite nasty.
Log attached as usual.
Amon Ott
--
Dr. Amon
that the main mds connection was
missing, it tried to reconnect and hung.
We do appreciate how much work you are doing to make Ceph production ready.
Thank you!
We really want to use Ceph in our product, because the design and experience
so far have shown a lot of potential.
Amon Ott
--
Dr. Amon Ott
On Thursday 08 December 2011 wrote Amon Ott:
On Wednesday 07 December 2011 wrote Sage Weil:
I pushed a patch to wip-d-lock that may fix this one, but unfortunately
don't have time to test this very carefully right now. Let us know if
that helps, or you can wait until next week
for the application to handle the error instead of blocking
forever without a chance to recover.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 24342336
10179 Berlin http://www.m-privacy.de
Amtsgericht Charlottenburg, HRB 84946
On Tuesday 06 December 2011 wrote Amon Ott:
Merged in and bug seems to be fixed. No more deadlock warnings today.
Unfortunately, I got another deadlock message in the log today. Full log of
one boot time is attached.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am
On Monday 05 December 2011 wrote Amon Ott:
On Friday 02 December 2011 wrote Sage Weil:
On Fri, 2 Dec 2011, Amon Ott wrote:
On Thursday 01 December 2011 you wrote:
On all four nodes of my test cluster, MDS crashes with a trace like
that in bug #1047. Example and ceph.conf attached
On Thursday 01 December 2011 wrote Amon Ott:
On Thursday 01 December 2011 wrote Sage Weil:
On Thu, 1 Dec 2011, Amon Ott wrote:
On Wednesday 30 November 2011 wrote Sage Weil:
I pushed a wip-i-ceph-lock branch to ceph-client.git that replaces
our (ab?)use of i_lock with a new
On Friday 02 December 2011 wrote Sage Weil:
On Fri, 2 Dec 2011, Amon Ott wrote:
On Thursday 01 December 2011 you wrote:
On all four nodes of my test cluster, MDS crashes with a trace like
that in bug #1047. Example and ceph.conf attached. Ceph server side is
from git master, last
Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 24342336
10179 Berlin http://www.m-privacy.de
Amtsgericht Charlottenburg, HRB 84946
Geschäftsführer:
Dipl.-Kfm. Holger Maczkowsky,
Roman Maczkowsky
GnuPG-Key-ID: 0x2DD3A649
On all four nodes of my test cluster, MDS crashes with a trace like that in
bug #1047. Example and ceph.conf attached. Ceph server side is from git
master, last commit ce6572273943ffdca4b7dc5344152d6c35106a2d.
MDS does not start on any node here, it reliably crashes with that assert.
Amon Ott
On Thursday 01 December 2011 wrote Sage Weil:
On Thu, 1 Dec 2011, Amon Ott wrote:
On Wednesday 30 November 2011 wrote Sage Weil:
I pushed a wip-i-ceph-lock branch to ceph-client.git that replaces our
(ab?)use of i_lock with a new i_ceph_lock in the ceph inode. This
avoids being bitten
Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 24342336
10179 Berlin http://www.m-privacy.de
Amtsgericht Charlottenburg, HRB 84946
Geschäftsführer:
Dipl.-Kfm. Holger Maczkowsky,
Roman Maczkowsky
GnuPG-Key-ID: 0x2DD3A649
Nov 30
to check.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 24342336
10179 Berlin http://www.m-privacy.de
Amtsgericht Charlottenburg, HRB 84946
Geschäftsführer:
Dipl.-Kfm. Holger Maczkowsky,
Roman Maczkowsky
GnuPG-Key-ID
On Thursday 03 November 2011 you wrote:
On Thu, Nov 3, 2011 at 05:02, Amon Ott a@m-privacy.de wrote:
Documentation recommends three monitors. In our special cluster
configuration, this would mean that if accidentially two nodes with
monitors fail (e.g. one in maintenance and one crashes
nodes. According to documentation, the monitor problem means that one failing
monitor kills the cluster whatever the number of defined monitors (1 or 2),
even if we have all data safely placed on both nodes.
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen
to
https://download.m-privacy.de/kern-full.log.bz2
Full ceph logging had been enabled as soon as possible, after boot and before
mounting ceph fs.
Also, the below patch may help us parse the output with multiple threads.
Put in the patch, new kernel packages building now.
Amon Ott
--
Dr. Amon
On Tuesday 25 October 2011 wrote Amon Ott:
On Tuesday 25 October 2011 wrote Amon Ott:
On Monday 24 October 2011 wrote Yehuda Sadeh Weinraub:
Also, the client logs could help shedding a light on the issue. You
should have dynamic debugging turned on (CONFIG_DYNAMIC_DEBUG), and
something
On Monday 24 October 2011 wrote Yehuda Sadeh Weinraub:
On Mon, Oct 24, 2011 at 3:39 AM, Amon Ott a@m-privacy.de wrote:
we have hit a kernel bug with current ceph-client master (commit
a2742a09568f81315e0f30021f29f14e7cd3924b), which I assume to be a Ceph
bug.
Is it easily reproducible
On Tuesday 25 October 2011 wrote Amon Ott:
On Monday 24 October 2011 wrote Yehuda Sadeh Weinraub:
Also, the client logs could help shedding a light on the issue. You
should have dynamic debugging turned on (CONFIG_DYNAMIC_DEBUG), and
something along the lines of:
# mount -t debugfs none
); */
#if BITS_PER_LONG == 32
- ino = ceph_ino_to_ino32(ino);
+ return ceph_ino_to_ino32(vino.ino);
+#else
+ return (ino_t)vino.ino;
#endif
- return ino;
}
Amon Ott
--
Dr. Amon Ott
m-privacy GmbH Tel: +49 30 24342334
Am Köllnischen Park 1Fax: +49 30 24342336
53 matches
Mail list logo
