Re: [ceph-users] rbd lock remove unable to parse address

2018-07-10 0:35 GMT+02:00 Jason Dillaman : > Is the link-local address of "fe80::219:99ff:fe9e:3a86%eth0" at least > present on the client computer you used? I would have expected the OSD to > determine the client address, so it's odd that it was able to get a > link-local address. > Yes, it is.

Re: [ceph-users] Rotating Cephx Keys

On 07/10/2018 11:41 AM, Graeme Gillies wrote: I think you are missing the part where if you update a key in ceph, in the space between that and when you update it in ovirt-engine any new connections to ceph by any ovirt nodes will fail Yes, this is should be seconds. But, actually startup will

Re: [ceph-users] Rotating Cephx Keys

On 10/07/18 14:37, Konstantin Shalygin wrote: >> If I >> want to rotate the keys for that I can simply do that ceph cluster side, >> but then I also need to do that on the client side (in my case virtual >> machine hypervisors). DUring this window (which might be tiny with >> decent tooling, but

Re: [ceph-users] Rotating Cephx Keys

If I want to rotate the keys for that I can simply do that ceph cluster side, but then I also need to do that on the client side (in my case virtual machine hypervisors). DUring this window (which might be tiny with decent tooling, but still non-zero) my clients can't do new connections to the cep

Re: [ceph-users] Erasure coding RBD pool for OpenStack Glance, Nova and Cinder

Does someone have used EC pools with OpenStack in production ? By chance, I found that link : https://www.reddit.com/r/ceph/comments/72yc9m/ceph_openstack_with_ec/ Yes, this good post. My configuration is: cinder.conf: [erasure-rbd-hdd] volume_driver = cinder.volume.drivers.rbd.RBDDriver

[ceph-users] Recovering from no quorum (2/3 monitors down) via 1 good monitor

Hi, I am running proxmox pve-5.1, with ceph luminous 12.2.4 as storage. I have been running on 3 monitors, up until an abrupt power outage, resulting in 2 monitors down and unable to start, while 1 monitor up but with no quorum. I tried extracting monmap from the good monitor and injecting it int

Re: [ceph-users] CephFS - How to handle "loaded dup inode" errors

While we're on this topic, could someone please explain to me what `cephfs-table-tool all reset inode` does? Does it only reset what the MDS has in its cache, and after starting up again, the MDS will read in new inode range from the metadata pool? If so, does it mean *before* we run `cephfs-

Re: [ceph-users] Rotating Cephx Keys

On Mon, Jul 9, 2018 at 4:57 PM Graeme Gillies wrote: > On 10/07/18 04:40, Gregory Farnum wrote: > > On Sun, Jul 8, 2018 at 6:06 PM Graeme Gillies wrote: > >> Hi, >> >> I was wondering how (if?) people handle rotating cephx keys while >> keeping cluster up/available. >> >> Part of meeting complia

Re: [ceph-users] Rotating Cephx Keys

On 10/07/18 04:40, Gregory Farnum wrote: > On Sun, Jul 8, 2018 at 6:06 PM Graeme Gillies > wrote: > > Hi, > > I was wondering how (if?) people handle rotating cephx keys while > keeping cluster up/available. > > Part of meeting compliance standards such

Re: [ceph-users] CephFS - How to handle "loaded dup inode" errors

We're affected by something like this right now (the dup inode causing MDS to crash via assert(!p) with add_inode(CInode) function). In terms of behaviours, shouldn't the MDS simply skip to the next available free inode in the event of a dup, than crashing the entire FS because of one file? Pro

Re: [ceph-users] rbd lock remove unable to parse address

Is the link-local address of "fe80::219:99ff:fe9e:3a86%eth0" at least present on the client computer you used? I would have expected the OSD to determine the client address, so it's odd that it was able to get a link-local address. On Mon, Jul 9, 2018 at 3:43 PM Kevin Olbrich wrote: > 2018-07-09

Re: [ceph-users] rbd lock remove unable to parse address

2018-07-09 21:25 GMT+02:00 Jason Dillaman : > BTW -- are you running Ceph on a one-node computer? I thought IPv6 > addresses starting w/ fe80 were link-local addresses which would probably > explain why an interface scope id was appended. The current IPv6 address > parser stops reading after it en

Re: [ceph-users] rbd lock remove unable to parse address

BTW -- are you running Ceph on a one-node computer? I thought IPv6 addresses starting w/ fe80 were link-local addresses which would probably explain why an interface scope id was appended. The current IPv6 address parser stops reading after it encounters a non hex, colon character [1]. On Mon, Jul

Re: [ceph-users] rbd lock remove unable to parse address

Is it possible to force-remove the lock or the image? Kevin 2018-07-09 21:14 GMT+02:00 Jason Dillaman : > Hmm ... it looks like there is a bug w/ RBD locks and IPv6 addresses since > it is failing to parse the address as valid. Perhaps it's barfing on the > "%eth0" scope id suffix within the add

Re: [ceph-users] rbd lock remove unable to parse address

Hmm ... it looks like there is a bug w/ RBD locks and IPv6 addresses since it is failing to parse the address as valid. Perhaps it's barfing on the "%eth0" scope id suffix within the address. On Mon, Jul 9, 2018 at 2:47 PM Kevin Olbrich wrote: > Hi! > > I tried to convert an qcow2 file to rbd an

Re: [ceph-users] luminous ceph-fuse with quotas breaks 'mount' and 'df'

Hi Greg, Am i reading this right that you've got a 1-*byte* quota but have gigabytes of data in the tree? I have no idea what that might do to the system, but it wouldn't totally surprise me if that was messing something up. Since <10KB definitely rounds towards 0... Yeah, that directory

[ceph-users] rbd lock remove unable to parse address

Hi! I tried to convert an qcow2 file to rbd and set the wrong pool. Immediately I stopped the transfer but the image is stuck locked: Previusly when that happened, I was able to remove the image after 30 secs. [root@vm2003 images1]# rbd -p rbd_vms_hdd lock list fpi_server02 There is 1 exclusive

Re: [ceph-users] Rotating Cephx Keys

On Sun, Jul 8, 2018 at 6:06 PM Graeme Gillies wrote: > Hi, > > I was wondering how (if?) people handle rotating cephx keys while > keeping cluster up/available. > > Part of meeting compliance standards such as PCI DSS is making sure that > data encryption keys and security credentials are rotated

Re: [ceph-users] Slow response while "tail -f" on cephfs

On Mon, Jul 9, 2018 at 12:46 AM Zhou Choury wrote: > Hi all > We mounted cephfs with ceph-fuse on two machines. We found that if a > process writing a log on node A, while "tail -f" it on node B will quite > slow, The mds server also complain like: > > 2018-07-09 15:10:35.516602 7f32fa0c2700 0

Re: [ceph-users] SSD for bluestore

bluestore doesn't have a journal like the filestore does, but there is the WAL (Write-Ahead Log) which is looks like a journal but works differently. You can (or must, depending or your needs) have SSDs to serve this WAL (and for Rocks DB). Regards, Webert Lima DevOps Engineer at MAV Tecnologia *

Re: [ceph-users] luminous ceph-fuse with quotas breaks 'mount' and 'df'

On Fri, Jul 6, 2018 at 10:30 AM Chad William Seys wrote: > Hi all, >I'm having a problem that when I mount cephfs with a quota in the > root mount point, no ceph-fuse appears in 'mount' and df reports: > > Filesystem 1K-blocks Used Available Use% Mounted on > ceph-fuse 0

[ceph-users] iSCSI SCST not working with Kernel 4.17.5

Hi, Just wondering if any of you managed to use SCST with kernel 4.17.5 ? Apparently SCST works only with kernel 3.10 Alternatively, is ceph-iscsi running properly with the newest kernel ? Installationa and configuration went well but accessing the LUN fail with the following error ".. kernel:

[ceph-users] Mimic 13.2.1 release date

Hi, Is there a release date for Mimic 13.2.1 yet? There are a few issues which currently make deploying with Mimic 13.2.0 a bit difficult, for example: - https://tracker.ceph.com/issues/24423 - https://github.com/ceph/ceph/pull/22393 Especially the first one makes it difficult. 13.2.1 would be

Re: [ceph-users] client.bootstrap-osd authentication error - which keyrin

2018-07-09 16:10 GMT+02:00 Thomas Roth : > Thanks, but doesn't work. > > It is always the subcommand > /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring > /var/lib/ceph/bootstrap-osd/ceph.keyring osd tree -f json > > (also 'ceph ... osd tree -i - osd new NEWID') > > which fails wi

Re: [ceph-users] client.bootstrap-osd authentication error - which keyrin

On Mon, Jul 9, 2018 at 10:10 AM, Thomas Roth wrote: > Thanks, but doesn't work. > > It is always the subcommand > /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring > /var/lib/ceph/bootstrap-osd/ceph.keyring osd tree -f json > > (also 'ceph ... osd tree -i - osd new NEWID') > > whi

Re: [ceph-users] client.bootstrap-osd authentication error - which keyrin

Thanks, but doesn't work. It is always the subcommand /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd tree -f json (also 'ceph ... osd tree -i - osd new NEWID') which fails with client.bootstrap-osd authentication error Of course,

Re: [ceph-users] FYI - Mimic segv in OSD

> On 9 Jul 2018, at 15.49, John Spray wrote: > > On Mon, Jul 9, 2018 at 2:37 PM Steffen Winther Sørensen > wrote: >> >> Dunno if this has been seen before so just for info, 1 in 24 OSD just did >> this: >> >> Jul 9 15:13:35 n4 ceph-osd: *** Caught signal (Segmentation fault) ** >> Jul 9 1

Re: [ceph-users] FYI - Mimic segv in OSD

On Mon, Jul 9, 2018 at 2:37 PM Steffen Winther Sørensen wrote: > > Dunno if this has been seen before so just for info, 1 in 24 OSD just did > this: > > Jul 9 15:13:35 n4 ceph-osd: *** Caught signal (Segmentation fault) ** > Jul 9 15:13:35 n4 ceph-osd: in thread 7ff209282700 thread_name:msgr-wo

Re: [ceph-users] luminous ceph-fuse with quotas breaks 'mount' and 'df'

On Fri, Jul 6, 2018 at 6:30 PM Chad William Seys wrote: > > Hi all, >I'm having a problem that when I mount cephfs with a quota in the > root mount point, no ceph-fuse appears in 'mount' and df reports: > > Filesystem 1K-blocks Used Available Use% Mounted on > ceph-fuse 0

[ceph-users] FYI - Mimic segv in OSD

Dunno if this has been seen before so just for info, 1 in 24 OSD just did this: Jul 9 15:13:35 n4 ceph-osd: *** Caught signal (Segmentation fault) ** Jul 9 15:13:35 n4 ceph-osd: in thread 7ff209282700 thread_name:msgr-worker-2 Jul 9 15:13:35 n4 kernel: msgr-worker-2[4697]: segfault at 0 ip 000

[ceph-users] Different write pools for RGW objects

Hi, I was wondering if I can have  different destination pools for the S3 objects uploaded to Ceph via RGW based on the object's size. For example : - smaller S3 objects (let's say smaller than 1MB) should go to a replicated pool - medium and big objects should go to a EC pool Is there an

Re: [ceph-users] fuse vs kernel client

Hi Manuel, My own experiences are that cephfs kernel client is significantly faster than fuse, however the fuse client is generally more reliable. If you need the extra speed of the kernel client on Centos, it may be worth using the ml kernel, as this gives you much more up to date cephfs support

[ceph-users] radosgw frontend : civetweb vs fastcgi

Hi: I see that civetweb is still using poll and multithread, compared with fastcgi, which one should I use ? Which one has better performance ? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph

Re: [ceph-users] Slow requests

On Mon, Jul 9, 2018 at 5:28 PM, Benjamin Naber wrote: > Hi @all, > > Problem seems to be solved, afther downgrading from Kernel 4.17.2 to > 3.10.0-862. > Anyone other have issues with newer Kernels and osd nodes? I'd suggest you pursue that with whoever supports the kernel exhibiting the problem

Re: [ceph-users] fuse vs kernel client

On 07/09/2018 10:18 AM, Manuel Sopena Ballesteros wrote: > FUSE is supposed to run slower. in our tests with ceph 11.2.x and 12.2.x clusters, cephfs-fuse is always around 10 times slower than kernel cephfs. Regards, Daniel ___ ceph-users mailing list ce

[ceph-users] fuse vs kernel client

Dear ceph community, I just installed ceph luminous in a small NVMe cluster for testing and I tested 2 clients: Client 1: VM running centos 7 Ceph client: kernel # cpus: 4 RAM: 16GB Fio test # sudo fio --name=xx --filename=/mnt/mycephfs/test.file3 --filesize=100G --iodepth=1 --rw=write --bs=4

[ceph-users] Slow response while "tail -f" on cephfs

Hi all We mounted cephfs with ceph-fuse on two machines. We found that if a process writing a log on node A, while "tail -f" it on node B will quite slow, The mds server also complain like: > 2018-07-09 15:10:35.516602 7f32fa0c2700 0 log_channel(cluster) log [WRN] : 2 > slow requests, 1 includ

[ceph-users] OT: Bad Sector Count - suggestions and experiences?

Hi, I apologize for the OT, but I hope some ceph users with bigger installations have a lot more experiences than the users reporting theire home problem (NAS with 2 disks … ) I saw a lot googling that topic. Luckily we had not as much hard disk failures as some coworkers, but now with more an