[ceph-users] minimal permission set for an rbd client

[cek+ceph]

I'm following the guide @ https://docs.ceph.com/en/latest/rbd/rados-rbd-cmds/
but I'm not following why would an `mgr` permission be required to have a 
functioning RBD client?

Thanks.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] unable to map device with krbd on el7 with ceph nautilus

[cek+ceph]

Hi. 

I've followed the installation guide and got nautilus 14.2.22 running on el7 
via https://download.ceph.com/rpm-nautilus/el7/x86_64/ yum repo. 
I'm now trying to map a device on an el7 and getting extremely weird errors:

# rbd info test1/blk1 --name client.testing-rw
rbd image 'blk1':
size 50 GiB in 12800 objects
order 22 (4 MiB objects)
snapshot_count: 0
id: 2e0929313a08e
block_name_prefix: rbd_data.2e0929313a08e
format: 2
features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
op_features: 
flags: 
create_timestamp: Fri Jul 23 15:59:12 2021
access_timestamp: Fri Jul 23 15:59:12 2021
modify_timestamp: Fri Jul 23 15:59:12 2021

# rbd device map test1/blk1 --name client.testing-rw
rbd: sysfs write failed
In some cases useful info is found in syslog - try "dmesg | tail".
rbd: map failed: (3) No such process

# dmesg | tail
[91885.624859] libceph: resolve 'name=testing-rw' (ret=-3): failed
[91885.624863] libceph: parse_ips bad ip 'name=testing-rw,key=client.testing-rw'

# modinfo rbd  |grep vers
filename:   
/lib/modules/3.10.0-1160.31.1.el7.x86_64/kernel/drivers/block/rbd.ko.xz
rhelversion:7.9
srcversion: 5386BBBD00C262C66CB81F5
vermagic:   3.10.0-1160.31.1.el7.x86_64 SMP mod_unload modversions 

Is that because the kernel module is too old? el7's base is supplying 
ceph-common-10.2.5.
Where can I get an updated kernel module for el7 and it would also be wonderful 
to have some form of warning in docs that user might encounter this problem.

Thanks.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: unable to map device with krbd on el7 with ceph nautilus

[cek+ceph]

Although I appreciate the responses, they have provided zero help solving this 
issue thus far.
It seems like the kernel module doesn't even get to the stage where it reads 
the attributes/features of the device. It doesn't know where to connect and, 
presumably, is confused by the options passed by userspace. 

Obviously, I have already tried "--user", "--name" and so on with similar 
messages in dmesg. I don't feel like downgrading to 10. Any way to make el7 
work with 14 userspace utils?

I have also just realized there was 1 message missing from provided logs, which 
somehow was logged by userspace(?) and not kernel. Here's the full log:

# rbd device map test1/blk1 --user testing-rw

Jul 26 05:33:53 xx key.dns_resolver[9147]: name=testing-rw: No address 
associated with name
Jul 26 05:33:53 xx kernel: libceph: resolve 'name=testing-rw' (ret=-3): failed
Jul 26 05:33:53 xx kernel: libceph: parse_ips bad ip 
'name=testing-rw,key=client.testing-rw'

# rbd info test1/blk1 --user testing-rw
works perfectly

Thanks.


On 7/24/21 12:47 PM, Marc wrote:
> 
> If you have the default kernel you can not use all these features. I think 
> even dmesg shows you something about that when mapping.
> 
> 
>> -Original Message-
>> From: cek+c...@deepunix.net
>> Sent: Friday, 23 July 2021 23:58
>> To: ceph-users@ceph.io
>> Subject: *SPAM* [ceph-users] unable to map device with krbd on
>> el7 with ceph nautilus
>>
>> Hi.
>>
>> I've followed the installation guide and got nautilus 14.2.22 running on

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: unable to map device with krbd on el7 with ceph nautilus

[cek+ceph]

Have found the problem. All this was caused by missing mon_host directive in 
ceph.conf. I have expected userspace to catch this, but it looks like it didn't 
care. 
We use DNS SRV in this cluster.

With mon_host directive reinstated, it was able to connect:
Jul 26 09:51:40 xx kernel: libceph: mon0 10.xx:6789 session established
Jul 26 09:51:40 xx kernel: libceph: client188721 fsid 
548a0823-815a-4ac5-a2e5-42cc7e8206ab
Jul 26 09:51:40 xx kernel: rbd: image blk1: image uses unsupported features: 
0x38

I'm wondering what happens in case this mon1 host goes down, will the kernel 
module go through the remaining mon directive addresses?

As re: strace, here you go:

# strace -f -e write -s 500  rbd device map test1/blk1 --user testing-rw
strace: Process 12962 attached
strace: Process 12963 attached
strace: Process 12964 attached
[pid 12964] write(7, " name=testing-rw,key=client.testing-rw test1 blk1 -", 51) 
= -1 ESRCH (No such process)
[pid 12964] write(6, "\375\377\377\377", 4) = 4
[pid 12961] write(2, "rbd: sysfs write failed", 23rbd: sysfs write failed 

[pid 12964] +++ exited with 0 +++
[pid 12961] <... write resumed>)= 23
[pid 12961] write(2, "\n", 1
)   = 1
strace: Process 12970 attached
strace: Process 12971 attached
strace: Process 12972 attached
[pid 12961] write(6, "c", 1)= 1
strace: Process 12973 attached
strace: Process 12974 attached
strace: Process 12975 attached
strace: Process 12976 attached
[pid 12961] write(9, "c", 1)= 1
[pid 12961] write(12, "c", 1)   = 1
[pid 12961] write(6, "c", 1)= 1
[pid 12971] write(6, "c", 1)= 1
[pid 12971] write(12, "c", 1)   = 1
[pid 12961] write(9, "c", 1)= 1
[pid 12976] +++ exited with 0 +++
[pid 12975] +++ exited with 0 +++
[pid 12961] write(6, "c", 1)= 1
[pid 12961] write(6, "c", 1)= 1
[pid 12961] write(9, "c", 1)= 1
[pid 12961] write(12, "c", 1)   = 1
[pid 12974] +++ exited with 0 +++
[pid 12973] +++ exited with 0 +++
[pid 12961] write(6, "c", 1)= 1
[pid 12961] write(9, "c", 1)= 1
[pid 12961] write(12, "c", 1)   = 1
strace: Process 12977 attached
[pid 12961] write(6, "c", 1)= 1
strace: Process 12978 attached
strace: Process 12979 attached
strace: Process 12980 attached
strace: Process 12981 attached
[pid 12961] write(9, "c", 1)= 1
[pid 12961] write(12, "c", 1)   = 1
[pid 12961] write(6, "c", 1)= 1
[pid 12971] write(12, "c", 1)   = 1
[pid 12971] write(6, "c", 1)= 1
[pid 12961] write(9, "c", 1)= 1
strace: Process 12982 attached
[pid 12961] write(9, "c", 1)= 1
strace: Process 12983 attached
strace: Process 12984 attached
[pid 12978] write(12, "c", 1)   = 1
strace: Process 12985 attached
strace: Process 12986 attached
[pid 12961] write(6, "c", 1)= 1
[pid 12984] write(6, "c", 1)= 1
[pid 12984] write(9, "c", 1)= 1
[pid 12984] write(9, "c", 1)= 1
[pid 12984] write(9, "c", 1)= 1
[pid 12984] write(9, "c", 1)= 1
strace: Process 12987 attached
strace: Process 12988 attached
[pid 12984] write(9, "c", 1)= 1
[pid 12984] write(9, "c", 1)= 1
[pid 12984] write(9, "c", 1)= 1
[pid 12984] write(12, "c", 1)   = 1
[pid 12984] write(9, "c", 1)= 1
strace: Process 12989 attached
strace: Process 12990 attached
[pid 12961] write(1, "In some cases useful info is found in syslog - try 
\"dmesg | tail\".\n", 67In some cases useful info is found in syslog - try 
"dmesg | tail".
) = 67
[pid 12990] +++ exited with 0 +++
[pid 12989] +++ exited with 0 +++
[pid 12984] +++ exited with 0 +++
[pid 12983] +++ exited with 0 +++
[pid 12961] write(6, "c", 1)= 1
[pid 12961] write(9, "c", 1)= 1
[pid 12961] write(12, "c", 1)   = 1
[pid 12961] write(6, "c", 1)= 1
[pid 12982] +++ exited with 0 +++
[pid 12961] write(12, "c", 1)   = 1
[pid 12961] write(9, "c", 1)= 1
[pid 12981] +++ exited with 0 +++
[pid 12980] +++ exited with 0 +++
[pid 12961] write(6, "c", 1)= 1
[pid 12961] write(6, "c", 1)= 1
[pid 12961] write(9, "c", 1)= 1
[pid 12961] write(12, "c", 1)   = 1
[pid 12979] +++ exited with 0 +++
[pid 12978] +++ exited with 0 +++
[pid 12961] write(6, "c", 1)= 1
[pid 12961] write(9, "c", 1)= 1
[pid 12961] write(12, "c", 1)   = 1
[pid 12977] +++ exited with 0 +++
[pid 12961] write(2, "rbd: map failed: ", 17rbd: map failed: ) = 17
[pid 12961] write(2, "(3) No such process", 19(3) No such process) = 19
[pid 12961] write(2, "\n", 1
)   = 1
[pid 12985] +++ exited with 0 +++
[pid 12986] +++ exited with 0 +++
[pid 12987] +++ exited with 0 +++
[pid 12988] +++ exited with 0 +++
[pid 12961] write(6, "c", 1)= 1
[pid 12970] +++ exited with 0 +++
[pid 12961] write(9, "c", 1)