global_init fails when only specifying monitor address
Hi,
I tried to connect to a small Ceph setup on my desktop without cephx and
that failed:
root@stack01:~# ceph -m wido-desktop.widodh.nl:6789 -s
global_init: unable to open config file.
root@stack01:~#
I however worked with:
root@stack01:~# ceph -m wido-desktop.widodh.nl:6789 -c /dev/null -s
2012-04-26 14:55:33.828524pg v148: 594 pgs: 594 active+clean; 0
bytes data, 7740 KB used, 70571 MB / 76800 MB avail
2012-04-26 14:55:33.829622 mds e1: 0/0/1 up
2012-04-26 14:55:33.836144 osd e14: 3 osds: 3 up, 3 in
2012-04-26 14:55:33.886429 log 2012-04-26 14:52:50.674430 osd.1
[2a00:f10:11c:ab:52e5:49ff:fec2:c976]:6807/28366 12 : [INF] 1.2b scrub ok
2012-04-26 14:55:33.892423 mon e1: 1 mons at
{desktop=[2a00:f10:11c:ab:52e5:49ff:fec2:c976]:6789/0}
root@stack01:~#
I quick look at global_init.cc showed me why this happened, it simply
looks for a configuration file to open and when it can't it fails.
But if a monitor address is set, a config file shouldn't be mandatory.
It could be accomplished rather simple by setting the flag
CINIT_FLAG_NO_DEFAULT_CONFIG_FILE if a mon_host has been set, but to do
that conf->parse_argv(args); should move a few lines up.
Comments? Thoughts?
Wido
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OSD died
Hi, Anyone have any idea how to fix this ? Can i just correct conflict data in osdmaps ? On Wed, Apr 25, 2012 at 3:42 PM, Tomasz Paszkowski wrote: > After removing pool snapshot I was trying to make self managed > snapshot and after reading source this was the root cause of this > problem. > > > On Wed, Apr 25, 2012 at 1:24 PM, Tomasz Paszkowski wrote: >> after upgrade to v0.45 stack trace is as follows: >> >> Program received signal SIGABRT, Aborted. >> [Switching to Thread 0x7fffeac55700 (LWP 11011)] >> 0x75ebb445 in raise () from /lib/x86_64-linux-gnu/libc.so.6 >> (gdb) bt >> #0 0x75ebb445 in raise () from /lib/x86_64-linux-gnu/libc.so.6 >> #1 0x75ebebab in abort () from /lib/x86_64-linux-gnu/libc.so.6 >> #2 0x7680969d in __gnu_cxx::__verbose_terminate_handler() () >> from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 >> #3 0x76807846 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 >> #4 0x76807873 in std::terminate() () >> from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 >> #5 0x7680796e in __cxa_throw () >> from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 >> #6 0x00695ec0 in ceph::__ceph_assert_fail ( >> assertion=0x80b1ed "_size >= 0", file=0x80a222 "./include/interval_set.h", >> line=382, >> func=0x81bf60 "void interval_set::erase(T, T) [with T = snapid_t]") >> at common/assert.cc:75 >> #7 0x005d1359 in erase (len=..., start=..., this=0xbe5738) >> at ./include/interval_set.h:382 >> #8 subtract (a=..., this=0xbe5738) at ./include/interval_set.h:404 >> #9 OSD::advance_map (this=0xbca000, t=..., tfin=) >> at osd/OSD.cc:3475 >> #10 0x005d33bf in OSD::handle_osd_map (this=0xbca000, m=0x2a4c800) >> at osd/OSD.cc:3272 >> #11 0x005d4c9b in OSD::_dispatch (this=0xbca000, m=0x2a4c800) >> at osd/OSD.cc:2780 >> ---Type to continue, or q to quit--- >> #12 0x005d52a5 in OSD::ms_dispatch (this=0xbca000, m=0x2a4c800) >> at osd/OSD.cc:2605 >> #13 0x0067a91b in ms_deliver_dispatch (m=0x2a4c800, this=0xba8680) >> at msg/Messenger.h:178 >> #14 SimpleMessenger::dispatch_entry (this=0xba8680) >> at msg/SimpleMessenger.cc:363 >> #15 0x00648f1d in SimpleMessenger::DispatchThread::entry ( >> this=) at msg/SimpleMessenger.h:560 >> #16 0x779c2e9a in start_thread () >> from /lib/x86_64-linux-gnu/libpthread.so.0 >> #17 0x75f774bd in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> #18 0x in ?? () >> >> >> On Wed, Apr 25, 2012 at 12:11 PM, Tomasz Paszkowski wrote: >>> osd dump is like this: >>> >>> pool 0 'data' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num >>> 768 pgp_num 768 lpg_num 2 lpgp_num 2 last_change 1 owner 0 >>> crash_replay_interval 45 >>> pool 1 'metadata' rep size 2 crush_ruleset 1 object_hash rjenkins >>> pg_num 768 pgp_num 768 lpg_num 2 lpgp_num 2 last_change 1 owner 0 >>> pool 2 'rbd' rep size 2 crush_ruleset 2 object_hash rjenkins pg_num >>> 768 pgp_num 768 lpg_num 2 lpgp_num 2 last_change 1 owner 0 >>> pool 9 'nova' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num >>> 2568 pgp_num 2568 lpg_num 0 lpgp_num 0 last_change 1435 owner >>> 18446744073709551615 >>> removed_snaps [1~1] >>> pool 10 'glance' rep size 2 crush_ruleset 0 object_hash rjenkins >>> pg_num 2568 pgp_num 2568 lpg_num 0 lpgp_num 0 last_change 132 owner >>> 18446744073709551615 >>> >>> >>> On Wed, Apr 25, 2012 at 11:04 AM, Tomasz Paszkowski >>> wrote: Hi, After making and removing snapshot from one of the pools, all of the osd in cluster are dying with log like below: 2012-04-25 11:01:00.938313 7f66694b9700 osd.1 1434 removing old osdmap epoch 966 2012-04-25 11:01:00.938330 7f66694b9700 osd.1 1434 removing old osdmap epoch 967 2012-04-25 11:01:00.938348 7f66694b9700 osd.1 1434 advance to epoch 1435 (<= newest 1470) 2012-04-25 11:01:00.939437 7f66694b9700 osd.1 1435 advance_map epoch 1435 1325 pgs 2012-04-25 11:01:00.939455 7f66694b9700 osd.1 1435 pool 0 removed snaps [], unchanged (snap_epoch = 0) 2012-04-25 11:01:00.939469 7f66694b9700 osd.1 1435 pool 1 removed snaps [], unchanged (snap_epoch = 0) 2012-04-25 11:01:00.939482 7f66694b9700 osd.1 1435 pool 2 removed snaps [], unchanged (snap_epoch = 0) ./include/interval_set.h: In function 'void interval_set::erase(T, T) [with T = snapid_t]' thread 7f66694b9700 time 2012-04-25 11:01:00.939509 ./include/interval_set.h: 382: FAILED assert(_size >= 0) ceph version 0.44.1 (commit:c89b7f22c8599eb974e75a2f7a5f855358199dee) 1: (OSD::advance_map(ObjectStore::Transaction&, C_Contexts*)+0x2971) [0x5cfb51] 2: (OSD::handle_osd_map(MOSDMap*)+0x193c) [0x5d162c] 3: (OSD::_dispatch(Message*)+0x2eb) [0x5d34fb] 4: (OSD::ms_dispatch(Message*)+0x129) [0x5d3a59] 5: (SimpleMessenger::dispatch_entry()+0x78b) [0x67513b] 6: (SimpleM
Re: global_init fails when only specifying monitor address
On Thu, 26 Apr 2012, Wido den Hollander wrote:
> Hi,
>
> I tried to connect to a small Ceph setup on my desktop without cephx and that
> failed:
>
> root@stack01:~# ceph -m wido-desktop.widodh.nl:6789 -s
> global_init: unable to open config file.
> root@stack01:~#
>
> I however worked with:
>
> root@stack01:~# ceph -m wido-desktop.widodh.nl:6789 -c /dev/null -s
> 2012-04-26 14:55:33.828524pg v148: 594 pgs: 594 active+clean; 0 bytes
> data, 7740 KB used, 70571 MB / 76800 MB avail
> 2012-04-26 14:55:33.829622 mds e1: 0/0/1 up
> 2012-04-26 14:55:33.836144 osd e14: 3 osds: 3 up, 3 in
> 2012-04-26 14:55:33.886429 log 2012-04-26 14:52:50.674430 osd.1
> [2a00:f10:11c:ab:52e5:49ff:fec2:c976]:6807/28366 12 : [INF] 1.2b scrub ok
> 2012-04-26 14:55:33.892423 mon e1: 1 mons at
> {desktop=[2a00:f10:11c:ab:52e5:49ff:fec2:c976]:6789/0}
> root@stack01:~#
>
> I quick look at global_init.cc showed me why this happened, it simply looks
> for a configuration file to open and when it can't it fails.
>
> But if a monitor address is set, a config file shouldn't be mandatory.
>
> It could be accomplished rather simple by setting the flag
> CINIT_FLAG_NO_DEFAULT_CONFIG_FILE if a mon_host has been set, but to do that
> conf->parse_argv(args); should move a few lines up.
>
> Comments? Thoughts?
I wonder if the simplest thing to do is:
- never error out on missing config in the default search path
- always error out on missing config if it was explicitly specified via
-c foo or CEPH_CONF in environment.
?
sage
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: global_init fails when only specifying monitor address
On Thursday, April 26, 2012 at 9:33 AM, Sage Weil wrote:
> On Thu, 26 Apr 2012, Wido den Hollander wrote:
> > Hi,
> >
> > I tried to connect to a small Ceph setup on my desktop without cephx and
> > that
> > failed:
> >
> > root@stack01:~# ceph -m wido-desktop.widodh.nl:6789
> > (http://wido-desktop.widodh.nl:6789) -s
> > global_init: unable to open config file.
> > root@stack01:~#
> >
> > I however worked with:
> >
> > root@stack01:~# ceph -m wido-desktop.widodh.nl:6789
> > (http://wido-desktop.widodh.nl:6789) -c /dev/null -s
> > 2012-04-26 14:55:33.828524 pg v148: 594 pgs: 594 active+clean; 0 bytes
> > data, 7740 KB used, 70571 MB / 76800 MB avail
> > 2012-04-26 14:55:33.829622 mds e1: 0/0/1 up
> > 2012-04-26 14:55:33.836144 osd e14: 3 osds: 3 up, 3 in
> > 2012-04-26 14:55:33.886429 log 2012-04-26 14:52:50.674430 osd.1
> > [2a00:f10:11c:ab:52e5:49ff:fec2:c976]:6807/28366 12 : [INF] 1.2b scrub ok
> > 2012-04-26 14:55:33.892423 mon e1: 1 mons at
> > {desktop=[2a00:f10:11c:ab:52e5:49ff:fec2:c976]:6789/0}
> > root@stack01:~#
> >
> > I quick look at global_init.cc (http://global_init.cc) showed me why this
> > happened, it simply looks
> > for a configuration file to open and when it can't it fails.
> >
> > But if a monitor address is set, a config file shouldn't be mandatory.
> >
> > It could be accomplished rather simple by setting the flag
> > CINIT_FLAG_NO_DEFAULT_CONFIG_FILE if a mon_host has been set, but to do that
> > conf->parse_argv(args); should move a few lines up.
> >
> > Comments? Thoughts?
>
> I wonder if the simplest thing to do is:
>
> - never error out on missing config in the default search path
> - always error out on missing config if it was explicitly specified via
> -c foo or CEPH_CONF in environment.
>
> ?
>
> sage
I think this is probably right. I think that we may even error out correctly if
we don't have values specified that we need, but we'll need to check that.
I'm working on similar stuff as I look at monitor cluster additions for Carl,
so I'll look at this today.
-Greg
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: global_init fails when only specifying monitor address
On 04/26/2012 06:57 PM, Greg Farnum wrote:
On Thursday, April 26, 2012 at 9:33 AM, Sage Weil wrote:
On Thu, 26 Apr 2012, Wido den Hollander wrote:
Hi,
I tried to connect to a small Ceph setup on my desktop without cephx and that
failed:
root@stack01:~# ceph -m wido-desktop.widodh.nl:6789
(http://wido-desktop.widodh.nl:6789) -s
global_init: unable to open config file.
root@stack01:~#
I however worked with:
root@stack01:~# ceph -m wido-desktop.widodh.nl:6789
(http://wido-desktop.widodh.nl:6789) -c /dev/null -s
2012-04-26 14:55:33.828524 pg v148: 594 pgs: 594 active+clean; 0 bytes
data, 7740 KB used, 70571 MB / 76800 MB avail
2012-04-26 14:55:33.829622 mds e1: 0/0/1 up
2012-04-26 14:55:33.836144 osd e14: 3 osds: 3 up, 3 in
2012-04-26 14:55:33.886429 log 2012-04-26 14:52:50.674430 osd.1
[2a00:f10:11c:ab:52e5:49ff:fec2:c976]:6807/28366 12 : [INF] 1.2b scrub ok
2012-04-26 14:55:33.892423 mon e1: 1 mons at
{desktop=[2a00:f10:11c:ab:52e5:49ff:fec2:c976]:6789/0}
root@stack01:~#
I quick look at global_init.cc (http://global_init.cc) showed me why this
happened, it simply looks
for a configuration file to open and when it can't it fails.
But if a monitor address is set, a config file shouldn't be mandatory.
It could be accomplished rather simple by setting the flag
CINIT_FLAG_NO_DEFAULT_CONFIG_FILE if a mon_host has been set, but to do that
conf->parse_argv(args); should move a few lines up.
Comments? Thoughts?
I wonder if the simplest thing to do is:
- never error out on missing config in the default search path
- always error out on missing config if it was explicitly specified via
-c foo or CEPH_CONF in environment.
?
sage
I think this is probably right. I think that we may even error out correctly if
we don't have values specified that we need, but we'll need to check that.
I agree, that would be indeed much easier :)
I'm working on similar stuff as I look at monitor cluster additions for Carl,
so I'll look at this today.
Thanks!
Wido
-Greg
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Release/branch naming; input requested
Ceph is maturing as a product. This brings us new challenges. Up to this point, we've been putting out a new release every 2-3 weeks, and only rarely updating an old release. Update releases like 0.42.2 are still a fairly rare occasion, and the the last number has never reached 4. However, as we get more and more customers and integrators deploying Ceph, we need to provide more stable branches with long term maintenance. I'm expecting, worst case, we'll be looking at something like this: - "oldstable": right now, 0.41 plus bugfixes only, because that's what deployed out there; updates will be rare - "stable": minor new features will be brought in, but we'll need to do extensive upgrade testing - "integration": this will be what our OpenStack Folsom integration will be against; including e.g. RBD layering - "latest": new release every 2-3 weeks On top of these, if you ask for the autobuilt repositories, we have: - "master": autobuilt all the time - several branches similar to "master", one per ceph.git branch, that sometimes are helpful in isolating a bug, or testing whether a code change fixes it Now, all of these are relative names. For example, once OpenStack Folsom is released, we'll need to open a new line of development for integrating the next version, branching it off the "latest" at a suitable point in time. The problem with these relative names is, if you run a system against "stable", and what "stable" points at changes, suddenly your old & reliable system thinks it needs to be upgraded. It's better to look up the current value of stable, and point the installation at that for updates. Combine this with people tending to be pretty bad at remembering specific numbers (I was a Debian Developer and I still can't remember which one is Debian 5.0!), and a new practice emerged: releases started getting codenames. Debian originally started giving their releases codenames from the Toy Story movies: hamm, slink, potato, woody, sarge, etch, lenny, squeeze, wheezy. Ubuntu uses largely similar infrastructure, but quickly improved on that by alphabetizing their codenames: breezy, dapper, edgy, feisty, gutsy, hardy, intrepid, jaunty, karmic, lucid, maverick, natty, oneiric, precise; you don't need to remember the order to know maverick came after lucid. The theme here is adjectives describing an alliterative animal, Natty Narwhal etc. OpenStack has picked the same alphabetical idea, using geographic names near where their per-release development summit is hosted: austin, bexar, cactus, diablo, essex, folsom. A company I used to work for picked names of mountains, largely choosing based on the magnificence of the upcoming release. By that logic, agel would have fewer new features than matterhorn. The same theme of mountains could also be used alphabetically. (Back at that point, a large part of that reasoning was that we left version numbering completely up to sales; engineering just did a look up of codename.buildnumber => version string, and we were able to change the version number of the product without rebuilding.) On the flip side, Ubuntu also picked up an idea of versioning releases based on the time of the release: YY.MM, as in 11.10, 12.04. This works really well for them, because they release only every 6 months, and they put a lot of effort into being on schedule, so it's always .04 or .10. Now, here are my actual questions: 1. What should the "relative" names of the branches be? "stable" vs "latest" etc. I especially don't like "integration", but I do see a time where it is not ready for "stable" but still needs to branch off of "latest". 2. Do we want to use cutesy codenames? Alphabetical? Based on what theme? 3. Do we want to use calendar based names? "I'm using Ceph branch 2012.04"? (Or spell it 2012-04 to avoid confusing with 0.41 style versions?) 3. What do we do with version numbers? With a 2-3 week iteration, we'll end up with something like 0.41.x, 0.56.x for Folsom integration (less than a year from now), and 0.57, 0.58 etc for "latest". 4. What will be worthy of 1.0? Is it when the distributed file system is solid? Getting out of 0.x would help with separating the different branches based on major numbers, but I fear that window has closed already. Your input is welcome. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Release/branch naming; input requested
On Thu, Apr 26, 2012 at 5:09 PM, Tommi Virtanen wrote: > Ceph is maturing as a product. This brings us new challenges. Up to > this point, we've been putting out a new release every 2-3 weeks, and > only rarely updating an old release. Update releases like 0.42.2 are > still a fairly rare occasion, and the the last number has never > reached 4. However, as we get more and more customers and integrators > deploying Ceph, we need to provide more stable branches with long term > maintenance. > > I'm expecting, worst case, we'll be looking at something like this: > > - "oldstable": right now, 0.41 plus bugfixes only, because that's what > deployed out there; updates will be rare > - "stable": minor new features will be brought in, but we'll need to > do extensive upgrade testing > - "integration": this will be what our OpenStack Folsom integration > will be against; including e.g. RBD layering > - "latest": new release every 2-3 weeks > > On top of these, if you ask for the autobuilt repositories, we have: > > - "master": autobuilt all the time > - several branches similar to "master", one per ceph.git branch, that > sometimes are helpful in isolating a bug, or testing whether a code > change fixes it > > Now, all of these are relative names. For example, once OpenStack > Folsom is released, we'll need to open a new line of development for > integrating the next version, branching it off the "latest" at a > suitable point in time. > > The problem with these relative names is, if you run a system against > "stable", and what "stable" points at changes, suddenly your old & > reliable system thinks it needs to be upgraded. It's better to look up > the current value of stable, and point the installation at that for > updates. Combine this with people tending to be pretty bad at > remembering specific numbers (I was a Debian Developer and I still > can't remember which one is Debian 5.0!), and a new practice emerged: > releases started getting codenames. > > Debian originally started giving their releases codenames from the Toy > Story movies: hamm, slink, potato, woody, sarge, etch, lenny, squeeze, > wheezy. > > Ubuntu uses largely similar infrastructure, but quickly improved on > that by alphabetizing their codenames: breezy, dapper, edgy, feisty, > gutsy, hardy, intrepid, jaunty, karmic, lucid, maverick, natty, > oneiric, precise; you don't need to remember the order to know > maverick came after lucid. The theme here is adjectives describing an > alliterative animal, Natty Narwhal etc. > > OpenStack has picked the same alphabetical idea, using geographic > names near where their per-release development summit is hosted: > austin, bexar, cactus, diablo, essex, folsom. > > A company I used to work for picked names of mountains, largely > choosing based on the magnificence of the upcoming release. By that > logic, agel would have fewer new features than matterhorn. The same > theme of mountains could also be used alphabetically. (Back at that > point, a large part of that reasoning was that we left version > numbering completely up to sales; engineering just did a look up of > codename.buildnumber => version string, and we were able to change the > version number of the product without rebuilding.) > > On the flip side, Ubuntu also picked up an idea of versioning releases > based on the time of the release: YY.MM, as in 11.10, 12.04. This > works really well for them, because they release only every 6 months, > and they put a lot of effort into being on schedule, so it's always > .04 or .10. > > > > Now, here are my actual questions: > > 1. What should the "relative" names of the branches be? "stable" vs > "latest" etc. I especially don't like "integration", but I do see a > time where it is not ready for "stable" but still needs to branch off > of "latest". reallyold old current next latest/experimental > > 2. Do we want to use cutesy codenames? Alphabetical? Based on what theme? Yes. Theme should be voted on. > > 3. Do we want to use calendar based names? "I'm using Ceph branch > 2012.04"? (Or spell it 2012-04 to avoid confusing with 0.41 style > versions?) No. I don't think it makes much sense. Not at this point at least. > > 3. What do we do with version numbers? With a 2-3 week iteration, > we'll end up with something like 0.41.x, 0.56.x for Folsom integration > (less than a year from now), and 0.57, 0.58 etc for "latest". We can keep those, they are completely orthogonal. These are exactly what they are: dev cycle numbers. I'm not too afraid of big numbers there, as they become uninteresting once you have the other naming scheme. They have the nice property of monotonically increasing which is useful internally. > > 4. What will be worthy of 1.0? Is it when the distributed file system > is solid? Getting out of 0.x would help with separating the different > branches based on major numbers, but I fear that window has closed > already. > Do we need to have 1.0? I'd say that for the first release that
Re: [PATCH] crush: include header for global symbols
From: H Hartley Sweeten Date: Tue, 24 Apr 2012 17:38:37 -0700 > Include the header to pickup the definitions of the global symbols. > > Quiets the following sparse warnings: > > warning: symbol 'crush_find_rule' was not declared. Should it be static? > warning: symbol 'crush_do_rule' was not declared. Should it be static? > > Signed-off-by: H Hartley Sweeten Applied. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
