[ceph-users] How to enable lazyio under kcephfs?

ceph: 14.2.x kernel: 4.15 In cephfs, due to the need for cache consistency, When a client is executing buffer IO, another client will hang when reading and writing the same file It seems that lazyio can solve this problem, lazyio allows multiple clients to execute buffer IO at the same time(relax

[ceph-users] Re: Only 2/5 mon services running

It looks like the second mon server was down from my reboot. Restarted and everything is functional again but I still can’t figure out why only 2 out of the 5 mon servers is down and won’t start. If they were functioning, I probably wouldn’t have noticing the cluster being down. Thanks -jerem

[ceph-users] Only 2/5 mon services running

In an attempt to troubleshoot why only 2/5 mon services were running, I believe I’ve broke something: [ceph: root@cn01 /]# ceph orch ls NAME PORTS RUNNING REFRESHED AGE PLACEMENT alertmanager 1/1 81s ago9d count:1 crash

[ceph-users] Re: Global Recovery Event

This seems to have recovered on its own. Thank you -jeremy > On Jun 7, 2021, at 5:44 PM, Neha Ojha wrote: > > On Mon, Jun 7, 2021 at 5:24 PM Jeremy Hansen > wrote: >> >> >> I’m seeing this in my health status: >> >> progress: >>Global Recovery Event (13h) >>

[ceph-users] Re: Global Recovery Event

On Mon, Jun 7, 2021 at 5:24 PM Jeremy Hansen wrote: > > > I’m seeing this in my health status: > > progress: > Global Recovery Event (13h) > [] (remaining: 5w) > > I’m not sure how this was initiated but this is a cluster with almost zero > objects. Is the

[ceph-users] Global Recovery Event

I’m seeing this in my health status: progress: Global Recovery Event (13h) [] (remaining: 5w) I’m not sure how this was initiated but this is a cluster with almost zero objects. Is there a way to halt this process? Why would it estimate 5 weeks to reco

[ceph-users] Failed OSD has 29 Slow MDS Ops.

Hello, Nautilus 14.2.16 I had an OSD go bad about 10 days ago. Apparently as it was going down some MDS ops got hung up waiting for it to come back. I was out of town for a couple days and found the OSD 'Down and Out' when I checked in. (Also, oddly, the cluster did not appear to initiate recov

[ceph-users] Re: Connect ceph to proxmox

Hi Istvan, June 7, 2021 11:54 AM, "Szabo, Istvan (Agoda)" wrote: > So the client is on 14.2.20 the cluster is on 14.2.21. Seems like the Debian > buster repo is missing > the 21 update? Best ask the Proxmox dev's about a 14.2.21 build. Or you could build it yourself, there is everything in the

[ceph-users] Re: Why you might want packages not containers for Ceph deployments

We have no one currently using containers for anything. *Therefore, we run old CEPH code to avoid them. If there was an option to not do containers on modern CEPH, that would be better for alot of people who don't want them. -Ed On 6/7/2021 2:54 AM, Eneko Lacunza wrote: Hi Marc, El 4/6/21 a

[ceph-users] Re: Turning on "compression_algorithm" old pool with 500TB usage

On Mon, Jun 07, 2021 at 06:22:07AM +0300, Konstantin Shalygin wrote: > The same. You need to rewrite all your data You can do that without user interaction too, if you recreate all affected OSDs. That way the data on the OSD is recreated and thus compressed because it is "new". Florian signat

[ceph-users] Cephfs root/boot?

Has anyone added the 'conf.d' modules (and on the centos/rhel/fedora world done the selinux work) so that initramfs/dracut can 'direct kernel boot' cephfs as a guest image root file system?  It took some work for the nfs folks to manage being the root filesystem. Harry _

[ceph-users] Running ISCSI with Ubuntu 18.04 OS

Dear all , IS it possible to configure and run ISCSI when you are deploying ceph using ansible running on ubuntu 18.04 OS? please help me to know and if possible provide helpful links on that. Best Regard Michel ___ ceph-users mailing list -- ceph-user

[ceph-users] cephfs objets without 'parent' xattr?

Hi, during an OS upgrade from Ubuntu 18.04 to 20.04 we seem to have triggered a bcache bug on three OSD hosts. These hosts are used with a 6+2 EC pool used with CephFS, so a number of PGs are affected by the bug. We were able to restart two of the three hosts (and will run some extra scrubs

[ceph-users] Debian buster nautilus 14.2.21 missing?

Hi, Am I doing something wrong or the 21 update is missing for buster? Thank you This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have r

[ceph-users] Re: Connect ceph to proxmox

So the client is on 14.2.20 the cluster is on 14.2.21. Seems like the Debian buster repo is missing the 21 update? Istvan Szabo Senior Infrastructure Engineer --- Agoda Services Co., Ltd. e: istvan.sz...@agoda.com ---

[ceph-users] slow ops at restarting OSDs (octopus)

Hello, I implemented a new cluster with 48 Nodes á 24 OSDs. I have a replicated pool with 4 replica. The crushrule distributes the replicas to different racks. With this cluster I tested a upgrade from Nautilis (14.2.20) to Octopus (15.2.13). The update itself worked well until I began the rest

[ceph-users] Re: Connect ceph to proxmox

Hi, Struggling with the global id reclaim. If it is set to true the clients can connect, but if it set to false as in the documentation is written, it will disconnect the old clients. The weird thing is that 1 client is already updated to the newest version, and when set to false, it disconnect

[ceph-users] Re: Why you might want packages not containers for Ceph deployments

Hi Marc, El 4/6/21 a las 16:39, Marc escribió: Do you use rbd images in containers that are residing on osd nodes? Does this give any problems? I used to have kernel mounted cephfs on a osd node, after a specific luminous release this was giving me problems. No, we use Ceph for VM storage. So

[ceph-users] Re: Debian buster nautilus 14.2.21 missing?

Hi, It is not available indeed. The package files are present but this was unintentional: nautilus 14.2.21 is not available on download.ceph.com. You need to get it from https://backports.debian.org/changes/buster-backports.html once it is packaged (it currently is 14.2.20). Cheers On 07/06/2

[ceph-users] Re: CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)

cephadm rm-daemon --name osd.29 on the node with the stale daemon did the trick. -jeremy > On Jun 7, 2021, at 2:24 AM, Jeremy Hansen wrote: > > Signed PGP part > So I found the failed daemon: > > [root@cn05 ~]# systemctl | grep 29 > > ● ceph-bfa2ad58-c049-11eb-9098-3c8cf8ed728d@osd.29.serv

[ceph-users] Re: CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)

So I found the failed daemon: [root@cn05 ~]# systemctl | grep 29 ● ceph-bfa2ad58-c049-11eb-9098-3c8cf8ed728d@osd.29.service loaded failed failedCeph osd.29 for bfa2ad58-c049-11eb-9098-3c8cf8ed728d But I’ve already replaced t

[ceph-users] Re: CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)

Thank you. So I see this: 2021-06-07T08:41:24.133493+ mgr.cn01.ceph.la1.clx.corp.xnkoft (mgr.224161) 1494 : cephadm [INF] Reconfiguring osd.29 (monmap changed)... 2021-06-07T08:44:37.650022+ mgr.cn01.ceph.la1.clx.corp.xnkoft (mgr.224161) 1592 : cephadm [INF] Reconfiguring osd.29 (monmap

[ceph-users] Re: CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)

Hello Jeremy Hansen, try: ceph log last cephadm or see files below /var/log/ceph/cephadm.log > On Jun 7, 2021, at 15:49, Jeremy Hansen wrote: > > What’s the proper way to track down where this error is coming from? Thanks. > > > 6/7/21 12:40:00 AM > [WRN] > [WRN] CEPHADM_FAILED_DAEMON: 1

[ceph-users] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)

What’s the proper way to track down where this error is coming from? Thanks. 6/7/21 12:40:00 AM [WRN] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 6/7/21 12:40:00 AM [WRN] Health detail: HEALTH_WARN 1 failed cephadm daemon(s) signature.asc Description: Message signed with O