[ceph-users] "unable to connect to cluster" after monitor IP change
gentooserver ~ # ceph-mon -i mon0 --extract-monmap /tmp/monmap
2018-04-06 15:38:10.863444 7f8aa2b72f80 -1 wrote monmap to /tmp/monmap
gentooserver ~ # monmaptool --print /tmp/monmap
monmaptool: monmap file /tmp/monmap
epoch 3
fsid a736559a-92d1-483e-9289-d2c7feed510f
last_changed 2018-04-06 14:53:12.892574
created 2018-04-06 14:52:18.190509
0: [2605:6000:1020:2056:7d79:3f08:ee64:2aa3]:6789/0 mon.mon0
This is the monmap I injected into my monitor.
gentooserver ~ # systemctl status ceph-mon@mon0
● ceph-mon@mon0.service - Ceph cluster monitor daemon
Loaded: loaded (/lib/systemd/system/ceph-mon@.service; indirect; vendor
preset: disabled)
Active: active (running) since Fri 2018-04-06 15:47:51 CDT; 16s ago
Main PID: 4362 (ceph-mon)
CGroup: /system.slice/system-ceph\x2dmon.slice/ceph-mon@mon0.service
└─4362 /usr/bin/ceph-mon -f --cluster ceph --id mon0 --setuser
ceph --setgroup ceph
Apr 06 15:47:51 gentooserver systemd[1]: Started Ceph cluster monitor
daemon.
Apr 06 15:47:51 gentooserver ceph-mon[4362]: 2018-04-06 15:47:51.841218
7f5824d44f80 -1 distro_detect - can't detect distro_version
gentooserver ~ # systemctl status ceph-mgr@mgr0
● ceph-mgr@mgr0.service - Ceph cluster manager daemon
Loaded: loaded (/lib/systemd/system/ceph-mgr@.service; indirect; vendor
preset: disabled)
Active: active (running) since Fri 2018-04-06 15:10:01 CDT; 38min ago
Main PID: 3807 (ceph-mgr)
CGroup: /system.slice/system-ceph\x2dmgr.slice/ceph-mgr@mgr0.service
└─3807 /usr/bin/ceph-mgr -f --cluster ceph --id mgr0 --setuser
ceph --setgroup ceph
Apr 06 15:10:01 gentooserver systemd[1]: Started Ceph cluster manager
daemon.
gentooserver ~ # systemctl status ceph-mds@mds0
● ceph-mds@mds0.service - Ceph metadata server daemon
Loaded: loaded (/lib/systemd/system/ceph-mds@.service; disabled; vendor
preset: disabled)
Drop-In: /etc/systemd/system/ceph-mds@.service.d
└─00gentoo.conf
Active: active (running) since Fri 2018-04-06 15:10:25 CDT; 38min ago
Main PID: 3827 (ceph-mds)
CGroup: /system.slice/system-ceph\x2dmds.slice/ceph-mds@mds0.service
└─3827 /usr/bin/ceph-mds -f --cluster ceph --id mds0 --setuser
ceph --setgroup ceph
Apr 06 15:10:25 gentooserver systemd[1]: Started Ceph metadata server
daemon.
Apr 06 15:10:25 gentooserver ceph-mds[3827]: starting mds.mds0 at -
All the daemons report they are running.
gentooserver ~ # ceph daemon mon.mon0 mon_status
{
"name": "mon0",
"rank": 0,
"state": "leader",
"election_epoch": 39,
"quorum": [
0
],
"features": {
"required_con": "153140804152475648",
"required_mon": [
"kraken",
"luminous"
],
"quorum_con": "2305244844532236283",
"quorum_mon": [
"kraken",
"luminous"
]
},
"outside_quorum": [],
"extra_probe_peers": [],
"sync_provider": [],
"monmap": {
"epoch": 3,
"fsid": "a736559a-92d1-483e-9289-d2c7feed510f",
"modified": "2018-04-06 14:53:12.892574",
"created": "2018-04-06 14:52:18.190509",
"features": {
"persistent": [
"kraken",
"luminous"
],
"optional": []
},
"mons": [
{
"rank": 0,
"name": "mon0",
"addr": "[2605:6000:1020:2056:7d79:3f08:ee64:2aa3]:6789/0",
"public_addr":
"[2605:6000:1020:2056:7d79:3f08:ee64:2aa3]:6789/0"
}
]
},
"feature_map": {
"mon": {
"group": {
"features": "0x1ffddff8eea4fffb",
"release": "luminous",
"num": 1
}
}
}
}
gentooserver ~ # cat /etc/ceph/ceph.conf
[global]
cluster = ceph
fsid = a736559a-92d1-483e-9289-d2c7feed510f
ms bind ipv6 = true
auth cluster required = none
auth service required = none
auth client required = none
mon host = gentooserver
mon addr = [2605:6000:1020:2056:7d79:3f08:ee64:2aa3]:6789
mon pg warn max per osd = 300
[mon]
mon initial members = mon0
mon host = gentooserver
mon addr = [2605:6000:1020:2056:7d79:3f08:ee64:2aa3]:6789
mon pg warn max per osd = 300
mon allow pool delete = true
[mon.mon0]
host = gentooserver
mon addr = [2605:6000:1020:2056:7d79:3f08:ee64:2aa3]:6789
[osd]
osd journal size = 1
osd crush chooseleaf type = 0
host = gentooserver
osd pool default size = 3
osd pool default min size = 2
[mds.mds0]
host = gentooserver
ceph -s times out:
gentooserver ~ # ceph -s
2018-04-06 15:58:29.861647 7f5e7f891700 0 monclient(hunting): authenticate
timed out after 300
2018-04-06 15:58:29.861672 7f5e7f891700 0 librados: client.admin
authentication error (110) Connection timed out
[errno 110] error connecting to the cluster
The cluster was working before the IP address changed. HELP
___
ceph-users mailing list
ceph-users@lists.ceph.com
[ceph-users] PG numbers don't add up?
I try to add a data pool: OSD_STAT USED AVAIL TOTAL HB_PEERSPG_SUM PRIMARY_PG_SUM 9 1076M 930G 931G [0,1,2,3,4,5,6,7,8]128 5 8 1076M 930G 931G [0,1,2,3,4,5,6,7,9]128 14 7 1076M 930G 931G [0,1,2,3,4,5,6,8,9]128 14 6 1076M 930G 931G [0,1,2,3,4,5,7,8,9]128 19 5 1076M 930G 931G [0,1,2,3,4,6,7,8,9]128 15 4 1076M 930G 931G [0,1,2,3,5,6,7,8,9]128 17 0 1076M 930G 931G [1,2,3,4,5,6,7,8,9]128 16 1 1076M 930G 931G [0,2,3,4,5,6,7,8,9]128 8 2 1076M 930G 931G [0,1,3,4,5,6,7,8,9]128 8 3 1076M 930G 931G [0,1,2,4,5,6,7,8,9]128 12 sum 10765M 9304G 9315G I try to add a metadata pool: sum 0 0 0 0 0 0 0 0 OSD_STAT USED AVAIL TOTAL HB_PEERSPG_SUM PRIMARY_PG_SUM 9 1076M 930G 931G [0,1,2,3,4,5,6,7,8] 73 73 8 1076M 930G 931G [0,1,2,3,4,5,6,7,9] 40 40 7 1076M 930G 931G [0,1,2,3,4,5,6,8,9] 56 56 6 1076M 930G 931G [0,1,2,3,4,5,7,8,9] 42 42 5 1076M 930G 931G [0,1,2,3,4,6,7,8,9] 54 54 4 1076M 930G 931G [0,1,2,3,5,6,7,8,9] 59 59 0 1076M 930G 931G [1,2,3,4,5,6,7,8,9] 38 38 1 1076M 930G 931G [0,2,3,4,5,6,7,8,9] 57 57 2 1076M 930G 931G [0,1,3,4,5,6,7,8,9] 45 45 3 1076M 930G 931G [0,1,2,4,5,6,7,8,9] 48 48 sum 10766M 9304G 9315G I try to add both pools: Error ERANGE: pg_num 128 size 10 would mean 2816 total pgs, which exceeds max 2000 (mon_max_pg_per_osd 200 * num_in_osds 10) That's over a thousand more PGs than both pools combined. Where are they coming from? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] (no subject)
Trying to create an OSD: gentooserver ~ # ceph-volume lvm create --data /dev/sdb Running command: ceph-authtool --gen-print-key Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new e70500fe-0d51-48c3-a607-414957886726 Running command: vgcreate --force --yes ceph-a736559a-92d1-483e-9289-d2c7feed510f /dev/sdb stdout: Volume group "ceph-a736559a-92d1-483e-9289-d2c7feed510f" successfully created Running command: lvcreate --yes -l 100%FREE -n osd-block-e70500fe-0d51-48c3-a607-414957886726 ceph-a736559a-92d1-483e-9289-d2c7feed510f stdout: Logical volume "osd-block-e70500fe-0d51-48c3-a607-414957886726" created. Running command: ceph-authtool --gen-print-key Running command: mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0 Running command: chown -R ceph:ceph /dev/dm-0 Running command: ln -s /dev/ceph-a736559a-92d1-483e-9289-d2c7feed510f/osd-block-e70500fe-0d51-48c3-a607-414957886726 /var/lib/ceph/osd/ceph-0/block Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap stderr: got monmap epoch 1 Running command: ceph-authtool /var/lib/ceph/osd/ceph-0/keyring --create-keyring --name osd.0 --add-key AQBEZqRalRoRCBAA03R6VshykLcZjMgQnFKDtg== stdout: creating /var/lib/ceph/osd/ceph-0/keyring stdout: added entity osd.0 auth auth(auid = 18446744073709551615 key=AQBEZqRalRoRCBAA03R6VshykLcZjMgQnFKDtg== with 0 caps) Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/ Running command: ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid e70500fe-0d51-48c3-a607-414957886726 --setuser ceph --setgroup ceph --> ceph-volume lvm prepare successful for: /dev/sdb Running command: ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/ceph-a736559a-92d1-483e-9289-d2c7feed510f/osd-block-e70500fe-0d51-48c3-a607-414957886726 --path /var/lib/ceph/osd/ceph-0 Running command: ln -snf /dev/ceph-a736559a-92d1-483e-9289-d2c7feed510f/osd-block-e70500fe-0d51-48c3-a607-414957886726 /var/lib/ceph/osd/ceph-0/block Running command: chown -R ceph:ceph /dev/dm-0 Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 Running command: systemctl enable ceph-volume@lvm-0-e70500fe-0d51-48c3-a607-414957886726 stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@lvm-0-e70500fe-0d51-48c3-a607-414957886726.service → /lib/systemd/system/ceph-volume@.service. Running command: systemctl start ceph-osd@0 stderr: Job for ceph-osd@0.service failed because the control process exited with error code. See "systemctl status ceph-osd@0.service" and "journalctl -xe" for details. --> Was unable to complete a new OSD, will rollback changes --> OSD will be fully purged from the cluster, because the ID was generated Running command: ceph osd purge osd.0 --yes-i-really-mean-it stderr: purged osd.0 --> RuntimeError: command returned non-zero exit status: 1 journalctl -xe -- Unit ceph-osd@0.service has begun starting up. Mar 10 17:14:34 gentooserver systemd[3977]: ceph-osd@0.service: Failed to execute command: No such file or directory Mar 10 17:14:34 gentooserver systemd[3977]: ceph-osd@0.service: Failed at step EXEC spawning /usr/lib/ceph/ceph-osd-prestart.sh: No such file or directory -- Subject: Process /usr/lib/ceph/ceph-osd-prestart.sh could not be executed -- Defined-By: systemd -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- The process /usr/lib/ceph/ceph-osd-prestart.sh could not be executed and failed. -- -- The error number returned by this process is 2. Mar 10 17:14:34 gentooserver systemd[1]: ceph-osd@0.service: Control process exited, code=exited status=203 Mar 10 17:14:34 gentooserver systemd[1]: ceph-osd@0.service: Failed with result 'exit-code'. Mar 10 17:14:34 gentooserver systemd[1]: Failed to start Ceph object storage daemon osd.0. -- Subject: Unit ceph-osd@0.service has failed -- Defined-By: systemd -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit ceph-osd@0.service has failed. -- -- The result is RESULT. Why is this file missing? Should I file a bug with my distro's packager? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Fwd: ceph-volume raw disks
-- Forwarded message -- From: Nathan Dehnel <ncdeh...@gmail.com> Date: Thu, Jan 25, 2018 at 9:49 PM Subject: Re: [ceph-users] ceph-volume raw disks To: David Turner <drakonst...@gmail.com> >Did you wipe all of the existing partitions and such first? I tried it both before and after creating an lvm physical partition. >Which version of ceph? 12.2.1 gentooserver ~ # ceph-volume lvm zap /dev/sdb usage: ceph-volume lvm [-h] ceph-volume lvm: error: unrecognized arguments: zap /dev/sdb ??? On Thu, Jan 25, 2018 at 9:42 PM, David Turner <drakonst...@gmail.com> wrote: > Did you wipe all of the existing partitions and such first? Which version > of ceph? The below commands are what I ran to re-add my osds as bluestore > after moving all data off of them. > > ceph-volume lvm zap /dev/sdb > ceph-volume lvm create --bluestore --data /dev/sdb > > On Thu, Jan 25, 2018 at 9:41 PM Nathan Dehnel <ncdeh...@gmail.com> wrote: > >> The doc at http://docs.ceph.com/docs/master/ceph-volume/lvm/prepare/ >> #ceph-volume-lvm-prepare says I can pass a physical device to >> ceph-volume. But when I try to do that: >> >> gentooserver ~ # ceph-volume lvm create --bluestore --data /dev/sdb >> usage: ceph-volume lvm create [-h] [--journal JOURNAL] --data DATA >> [--journal-size GB] [--bluestore] >> [--filestore] >> [--osd-id OSD_ID] [--osd-fsid OSD_FSID] >> ceph-volume lvm create: error: Logical volume must be specified as >> 'volume_group/logical_volume' but got: /dev/sdb >> >> Am I missing something? >> ___ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] ceph-volume raw disks
The doc at http://docs.ceph.com/docs/master/ceph-volume/lvm/prepare/#ceph-volume-lvm-prepare says I can pass a physical device to ceph-volume. But when I try to do that: gentooserver ~ # ceph-volume lvm create --bluestore --data /dev/sdb usage: ceph-volume lvm create [-h] [--journal JOURNAL] --data DATA [--journal-size GB] [--bluestore] [--filestore] [--osd-id OSD_ID] [--osd-fsid OSD_FSID] ceph-volume lvm create: error: Logical volume must be specified as 'volume_group/logical_volume' but got: /dev/sdb Am I missing something? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] ceph command hangs
11:11.984458 7f9579ffb700 1 --
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:0/285566692 -->
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789/0 -- mon_command({"prefix":
"get_command_descriptions"} v 0) v1 -- 0x7f955c0041d0 con 0
2018-01-18 21:11:11.984678 7f9579ffb700 1 --
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:0/285566692 <== mon.0
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789/0 2 mgrmap(e 1) v1
103+0+0 (706778617 0 0) 0x7f956c000fd0 con 0x7f9560007290
^CError EINTR: problem getting command descriptions from mon.
2018-01-18 21:11:41.999310 7f9571ffb700 1 --
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:0/285566692 >>
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789/0 conn(0x7f9560007290 :-1
s=STATE_OPEN pgs=5 cs=1 l=1).mark_down
2018-01-18 21:11:41.999382 7f9571ffb700 1 --
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:0/285566692 shutdown_connections
2018-01-18 21:11:41.999464 7f9571ffb700 1 --
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:0/285566692 shutdown_connections
2018-01-18 21:11:41.999488 7f9571ffb700 1 --
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:0/285566692 wait complete.
2018-01-18 21:11:41.999494 7f9571ffb700 1 --
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:0/285566692 >>
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:0/285566692 conn(0x7f957c118c90 :-1
s=STATE_NONE pgs=0 cs=0 l=0).mark_down
To my knowledge there are no firewall settings that would cause problems.
I'm running ceph -s on the monitor node.
On Tue, Jan 16, 2018 at 8:05 PM, Jean-Charles Lopez <jelo...@redhat.com>
wrote:
> Hi Nathan,
>
> I would have place the mon_host parameter and assigned it the IP address
> for your monitor host in the global section so that the client (ceph -s
> command).
>
> Have you also checked your firewall setup on your MON box?
>
> To help diagnose you can also use ceph -s --debug-ms=1 so you can follow
> the network exchange between your client machine and the MON.
>
> Regards
> JC
>
>
> On Jan 16, 2018, at 13:24, Nathan Dehnel <ncdeh...@gmail.com> wrote:
>
> I'm doing a manual setup following http://docs.ceph.
> com/docs/master/install/manual-deployment/
>
> The ceph command hangs until I kill it. I have 1 monitor service started.
> ==
> gentooserver ~ # ceph -s
> ^CError EINTR: problem getting command descriptions from mon.
> =
> gentooserver ~ # emerge -pqv ceph
> [ebuild R ] sys-cluster/ceph-12.2.1 USE="mgr radosgw ssl systemd
> tcmalloc -babeltrace -cephfs -fuse -jemalloc -ldap -lttng -nss -static-libs
> {-test} -xfs -zfs" CPU_FLAGS_X86="sse sse2 sse3 sse4_1 sse4_2 ssse3"
> PYTHON_TARGETS="python2_7 python3_5 -python3_4 -python3_6"
> ==
> gentooserver ~ # cat /etc/ceph/ceph.conf
> [global]
> cluster = ceph
> fsid = a736559a-92d1-483e-9289-d2c7feed510f
> ms bind ipv6 = true
> #public network = 2001:1c:d64b:91c5:3a84:dfce:8546:9982/0
> auth cluster required = none
> auth service required = none
> auth client required = none
>
> [mon]
> mon initial members = mon0
> mon host = gentooserver
> mon addr = [2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789
>
>
> [mon.mon0]
> host = gentooserver
> mon addr = [2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789
>
> [osd]
> osd journal size = 1
> osd crush chooseleaf type = 0
> ==
> gentooserver ~ # monmaptool --print /tmp/monmap
> monmaptool: monmap file /tmp/monmap
> epoch 0
> fsid a736559a-92d1-483e-9289-d2c7feed510f
> last_changed 2018-01-14 16:50:59.838277
> created 2018-01-14 16:50:59.838277
> 0: [2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789/0 mon.mon0
> ==
> gentooserver ~ # systemctl status ceph-mon@mon0 | cat
> ● ceph-mon@mon0.service - Ceph cluster monitor daemon
>Loaded: loaded (/usr/lib/systemd/system/ceph-mon@.service; enabled;
> vendor preset: disabled)
>Active: active (running) since Tue 2018-01-16 14:50:18 CST; 17min ago
> Main PID: 75938 (ceph-mon)
>CGroup: /system.slice/system-ceph\x2dmon.slice/ceph-mon@mon0.service
>└─75938 /usr/bin/ceph-mon -f --cluster ceph --id mon0 --setuser
> ceph --setgroup ceph
>
> Jan 16 14:50:18 gentooserver systemd[1]: Started Ceph cluster monitor
> daemon.
> Jan 16 14:50:18 gentooserver ceph-mon[75938]: 2018-01-16 14:50:18.977494
> 7ff07d4cef80 -1 distro_detect - can't detect distro_version
> ===
> gentooserver ~ # cat /var/log/ceph/ceph.log
> 2018-01-16 14:50:18.977541 mon.mon0 mon.0
> [2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789/0
> 1 : cluster [INF] mon.mon0@0 won leader election with quorum 0
> 2018-01
[ceph-users] ceph command hangs
I'm doing a manual setup following
http://docs.ceph.com/docs/master/install/manual-deployment/
The ceph command hangs until I kill it. I have 1 monitor service started.
==
gentooserver ~ # ceph -s
^CError EINTR: problem getting command descriptions from mon.
=
gentooserver ~ # emerge -pqv ceph
[ebuild R ] sys-cluster/ceph-12.2.1 USE="mgr radosgw ssl systemd
tcmalloc -babeltrace -cephfs -fuse -jemalloc -ldap -lttng -nss -static-libs
{-test} -xfs -zfs" CPU_FLAGS_X86="sse sse2 sse3 sse4_1 sse4_2 ssse3"
PYTHON_TARGETS="python2_7 python3_5 -python3_4 -python3_6"
==
gentooserver ~ # cat /etc/ceph/ceph.conf
[global]
cluster = ceph
fsid = a736559a-92d1-483e-9289-d2c7feed510f
ms bind ipv6 = true
#public network = 2001:1c:d64b:91c5:3a84:dfce:8546:9982/0
auth cluster required = none
auth service required = none
auth client required = none
[mon]
mon initial members = mon0
mon host = gentooserver
mon addr = [2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789
[mon.mon0]
host = gentooserver
mon addr = [2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789
[osd]
osd journal size = 1
osd crush chooseleaf type = 0
==
gentooserver ~ # monmaptool --print /tmp/monmap
monmaptool: monmap file /tmp/monmap
epoch 0
fsid a736559a-92d1-483e-9289-d2c7feed510f
last_changed 2018-01-14 16:50:59.838277
created 2018-01-14 16:50:59.838277
0: [2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789/0 mon.mon0
==
gentooserver ~ # systemctl status ceph-mon@mon0 | cat
● ceph-mon@mon0.service - Ceph cluster monitor daemon
Loaded: loaded (/usr/lib/systemd/system/ceph-mon@.service; enabled;
vendor preset: disabled)
Active: active (running) since Tue 2018-01-16 14:50:18 CST; 17min ago
Main PID: 75938 (ceph-mon)
CGroup: /system.slice/system-ceph\x2dmon.slice/ceph-mon@mon0.service
└─75938 /usr/bin/ceph-mon -f --cluster ceph --id mon0 --setuser
ceph --setgroup ceph
Jan 16 14:50:18 gentooserver systemd[1]: Started Ceph cluster monitor
daemon.
Jan 16 14:50:18 gentooserver ceph-mon[75938]: 2018-01-16 14:50:18.977494
7ff07d4cef80 -1 distro_detect - can't detect distro_version
===
gentooserver ~ # cat /var/log/ceph/ceph.log
2018-01-16 14:50:18.977541 mon.mon0 mon.0
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789/0 1 : cluster [INF] mon.mon0@0
won leader election with quorum 0
2018-01-16 14:50:18.977656 mon.mon0 mon.0
[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789/0 2 : cluster [INF] monmap e1:
1 mons at {mon0=[2001:1c:d64b:91c5:3a84:dfce:8546:9982]:6789/0}
---
gentooserver ~ # cat /var/log/ceph/ceph-mon.mon0.log
2018-01-16 14:50:18.760533 7ff07d4cef80 0 set uid:gid to 110:239
(ceph:ceph)
2018-01-16 14:50:18.760549 7ff07d4cef80 0 ceph version 12.2.1
(3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable), process
(unknown), pid 75938
2018-01-16 14:50:18.760591 7ff07d4cef80 0 pidfile_write: ignore empty
--pid-file
2018-01-16 14:50:18.765642 7ff07d4cef80 0 load: jerasure load: lrc load:
isa
2018-01-16 14:50:18.765702 7ff07d4cef80 0 set rocksdb option compression
= kNoCompression
2018-01-16 14:50:18.765709 7ff07d4cef80 0 set rocksdb option
write_buffer_size = 33554432
2018-01-16 14:50:18.765722 7ff07d4cef80 0 set rocksdb option compression
= kNoCompression
2018-01-16 14:50:18.765726 7ff07d4cef80 0 set rocksdb option
write_buffer_size = 33554432
2018-01-16 14:50:18.765798 7ff07d4cef80 4 rocksdb: RocksDB version: 5.4.0
2018-01-16 14:50:18.765804 7ff07d4cef80 4 rocksdb: Git sha
rocksdb_build_git_sha:@0@
2018-01-16 14:50:18.765806 7ff07d4cef80 4 rocksdb: Compile date Dec 14 2017
2018-01-16 14:50:18.765808 7ff07d4cef80 4 rocksdb: DB SUMMARY
2018-01-16 14:50:18.765837 7ff07d4cef80 4 rocksdb: CURRENT file: CURRENT
2018-01-16 14:50:18.765840 7ff07d4cef80 4 rocksdb: IDENTITY file: IDENTITY
2018-01-16 14:50:18.765843 7ff07d4cef80 4 rocksdb: MANIFEST file:
MANIFEST-11 size: 210 Bytes
2018-01-16 14:50:18.765845 7ff07d4cef80 4 rocksdb: SST files in
/var/lib/ceph/mon/ceph-mon0/store.db dir, Total Num: 3, files: 04.sst
07.sst 10.sst
2018-01-16 14:50:18.765846 7ff07d4cef80 4 rocksdb: Write Ahead Log file in
/var/lib/ceph/mon/ceph-mon0/store.db: 12.log size: 9161 ;
2018-01-16 14:50:18.765847 7ff07d4cef80 4 rocksdb:
Options.error_if_exists: 0
2018-01-16 14:50:18.765848 7ff07d4cef80 4 rocksdb:
Options.create_if_missing: 0
2018-01-16 14:50:18.765848 7ff07d4cef80 4 rocksdb:
Options.paranoid_checks: 1
2018-01-16 14:50:18.765849 7ff07d4cef80 4 rocksdb:
Options.env: 0x55bc0ff83a60
2018-01-16 14:50:18.765849 7ff07d4cef80 4 rocksdb:
Options.info_log: 0x55bc10a53ae0
2018-01-16 14:50:18.765850 7ff07d4cef80 4 rocksdb:
Re: [ceph-users] permission denied, unable to bind socket
Ok I fixed the address error. The service is able to start now. ceph -s hangs though. gentooserver ~ # ceph -s ^CError EINTR: problem getting command descriptions from mon. I'm not sure how to fix the permissions issue. /var/run/ceph is a temporary directory so I can't just chown it. On Sat, Jan 6, 2018 at 7:40 PM, Nathan Dehnel <ncdeh...@gmail.com> wrote: > So I'm following the guide at http://docs.ceph.com/docs/ > master/install/manual-deployment/ > > ceph-mon@gentooserver.service fails. > > Jan 06 19:12:40 gentooserver systemd[1]: Started Ceph cluster monitor > daemon. > Jan 06 19:12:41 gentooserver ceph-mon[2674]: warning: unable to create > /var/run/ceph: (13) Permission denied > Jan 06 19:12:41 gentooserver ceph-mon[2674]: 2018-01-06 19:12:41.000507 > 7f3163008f80 -1 asok(0x563d0f97d2c0) AdminSocketConfigObs::init: failed: > AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to > '/var/run/ceph/ceph-mon.gentooserver.asok': (2) No such file or directory > Jan 06 19:12:41 gentooserver ceph-mon[2674]: 2018-01-06 19:12:41.152781 > 7f3163008f80 -1 Processor -- bind unable to bind to > [2001:1c:d64b:91c5:3a84:dfce:8546:998]:6789/0: (99) Cannot assign > requested address > Jan 06 19:12:41 gentooserver ceph-mon[2674]: 2018-01-06 19:12:41.152789 > 7f3163008f80 -1 Processor -- bind was unable to bind. Trying again in 5 > seconds > Jan 06 19:12:46 gentooserver ceph-mon[2674]: 2018-01-06 19:12:46.152982 > 7f3163008f80 -1 Processor -- bind unable to bind to > [2001:1c:d64b:91c5:3a84:dfce:8546:998]:6789/0: (99) Cannot assign > requested address > Jan 06 19:12:46 gentooserver ceph-mon[2674]: 2018-01-06 19:12:46.152996 > 7f3163008f80 -1 Processor -- bind was unable to bind. Trying again in 5 > seconds > Jan 06 19:12:51 gentooserver ceph-mon[2674]: 2018-01-06 19:12:51.153218 > 7f3163008f80 -1 Processor -- bind unable to bind to > [2001:1c:d64b:91c5:3a84:dfce:8546:998]:6789/0: (99) Cannot assign > requested address > Jan 06 19:12:51 gentooserver ceph-mon[2674]: 2018-01-06 19:12:51.153244 > 7f3163008f80 -1 Processor -- bind was unable to bind after 3 attempts: > (99) Cannot assign requested address > Jan 06 19:12:51 gentooserver ceph-mon[2674]: 2018-01-06 19:12:51.153250 > 7f3163008f80 -1 unable to bind monitor to [2001:1c:d64b:91c5:3a84:dfce: > 8546:998]:6789/0 > Jan 06 19:12:51 gentooserver systemd[1]: ceph-mon@gentooserver.service: > Main process exited, code=exited, status=1/FAILURE > Jan 06 19:12:51 gentooserver systemd[1]: ceph-mon@gentooserver.service: > Unit entered failed state. > Jan 06 19:12:51 gentooserver systemd[1]: ceph-mon@gentooserver.service: > Failed with result 'exit-code'. > Jan 06 19:13:01 gentooserver systemd[1]: ceph-mon@gentooserver.service: > Service hold-off time over, scheduling restart. > Jan 06 19:13:01 gentooserver systemd[1]: Stopped Ceph cluster monitor > daemon. > Jan 06 19:13:01 gentooserver systemd[1]: ceph-mon@gentooserver.service: > Start request repeated too quickly. > Jan 06 19:13:01 gentooserver systemd[1]: Failed to start Ceph cluster > monitor daemon. > Jan 06 19:13:01 gentooserver systemd[1]: ceph-mon@gentooserver.service: > Unit entered failed state. > Jan 06 19:13:01 gentooserver systemd[1]: ceph-mon@gentooserver.service: > Failed with result 'exit-code'. > > cat /etc/ceph/ceph.conf > [global] > fsid = a736559a-92d1-483e-9289-d2c7feed510f > ms bind ipv6 = true > mon initial members = gentooserver > > [mon.mona] > host = gentooserver > mon addr = [2001:1c:d64b:91c5:3a84:dfce:8546:9982] > > [osd] > osd journal size = 1 > > I'm not sure if the problem is the permissions error, or the IP address > appearing to get truncated in the output, or both. > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] permission denied, unable to bind socket
So I'm following the guide at http://docs.ceph.com/docs/master/install/manual-deployment/ ceph-mon@gentooserver.service fails. Jan 06 19:12:40 gentooserver systemd[1]: Started Ceph cluster monitor daemon. Jan 06 19:12:41 gentooserver ceph-mon[2674]: warning: unable to create /var/run/ceph: (13) Permission denied Jan 06 19:12:41 gentooserver ceph-mon[2674]: 2018-01-06 19:12:41.000507 7f3163008f80 -1 asok(0x563d0f97d2c0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-mon.gentooserver.asok': (2) No such file or directory Jan 06 19:12:41 gentooserver ceph-mon[2674]: 2018-01-06 19:12:41.152781 7f3163008f80 -1 Processor -- bind unable to bind to [2001:1c:d64b:91c5:3a84:dfce:8546:998]:6789/0: (99) Cannot assign requested address Jan 06 19:12:41 gentooserver ceph-mon[2674]: 2018-01-06 19:12:41.152789 7f3163008f80 -1 Processor -- bind was unable to bind. Trying again in 5 seconds Jan 06 19:12:46 gentooserver ceph-mon[2674]: 2018-01-06 19:12:46.152982 7f3163008f80 -1 Processor -- bind unable to bind to [2001:1c:d64b:91c5:3a84:dfce:8546:998]:6789/0: (99) Cannot assign requested address Jan 06 19:12:46 gentooserver ceph-mon[2674]: 2018-01-06 19:12:46.152996 7f3163008f80 -1 Processor -- bind was unable to bind. Trying again in 5 seconds Jan 06 19:12:51 gentooserver ceph-mon[2674]: 2018-01-06 19:12:51.153218 7f3163008f80 -1 Processor -- bind unable to bind to [2001:1c:d64b:91c5:3a84:dfce:8546:998]:6789/0: (99) Cannot assign requested address Jan 06 19:12:51 gentooserver ceph-mon[2674]: 2018-01-06 19:12:51.153244 7f3163008f80 -1 Processor -- bind was unable to bind after 3 attempts: (99) Cannot assign requested address Jan 06 19:12:51 gentooserver ceph-mon[2674]: 2018-01-06 19:12:51.153250 7f3163008f80 -1 unable to bind monitor to [2001:1c:d64b:91c5:3a84:dfce:8546:998]:6789/0 Jan 06 19:12:51 gentooserver systemd[1]: ceph-mon@gentooserver.service: Main process exited, code=exited, status=1/FAILURE Jan 06 19:12:51 gentooserver systemd[1]: ceph-mon@gentooserver.service: Unit entered failed state. Jan 06 19:12:51 gentooserver systemd[1]: ceph-mon@gentooserver.service: Failed with result 'exit-code'. Jan 06 19:13:01 gentooserver systemd[1]: ceph-mon@gentooserver.service: Service hold-off time over, scheduling restart. Jan 06 19:13:01 gentooserver systemd[1]: Stopped Ceph cluster monitor daemon. Jan 06 19:13:01 gentooserver systemd[1]: ceph-mon@gentooserver.service: Start request repeated too quickly. Jan 06 19:13:01 gentooserver systemd[1]: Failed to start Ceph cluster monitor daemon. Jan 06 19:13:01 gentooserver systemd[1]: ceph-mon@gentooserver.service: Unit entered failed state. Jan 06 19:13:01 gentooserver systemd[1]: ceph-mon@gentooserver.service: Failed with result 'exit-code'. cat /etc/ceph/ceph.conf [global] fsid = a736559a-92d1-483e-9289-d2c7feed510f ms bind ipv6 = true mon initial members = gentooserver [mon.mona] host = gentooserver mon addr = [2001:1c:d64b:91c5:3a84:dfce:8546:9982] [osd] osd journal size = 1 I'm not sure if the problem is the permissions error, or the IP address appearing to get truncated in the output, or both. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] ceph.conf not found
Hey, I get this error:
gentooserver ~ # ceph -s
2018-01-04 14:38:35.390154 7f0a6bae8700 -1 Errors while parsing config file!
2018-01-04 14:38:35.390157 7f0a6bae8700 -1 parse_file: cannot open
/etc/ceph/ceph.conf: (2) No such file or directory
2018-01-04 14:38:35.390158 7f0a6bae8700 -1 parse_file: cannot open
~/.ceph/ceph.conf: (2) No such file or directory
2018-01-04 14:38:35.390158 7f0a6bae8700 -1 parse_file: cannot open
ceph.conf: (2) No such file or directory
Error initializing cluster client: ObjectNotFound('error calling
conf_read_file',)
I don't have a ceph.conf because my cluster name is "home" and the guide at
http://docs.ceph.com/docs/master/install/manual-deployment/ says to name
the configuration file after the cluster name. What should I do to resolve
this?
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
