[ceph-users] Re: mds slow request with “failed to authpin, subtree is being exported"

Thanks a lot, Xiubo! we already set ‘mds_bal_interval’ to 0. and the slow mds seems decrease. But somehow we still see mds complain slow request. and from mds log , can see “slow request *** seconds old, received at 2023-12-04T…: internal op exportdir:mds.* currently acquired locks” so our qu

[ceph-users] Re: How to identify the index pool real usage?

Hi, A flash system needs free space to work efficiently. Hence my hypothesis that fully allocated disks need to be notified of free blocks (trim) Cordialement, *David CASIER* Le

[ceph-users] Re: How to identify the index pool real usage?

These values shouldn't be true to be able to do triming? "bdev_async_discard": "false", "bdev_enable_discard": "false", Istvan Szabo Staff Infrastructure Engineer --- Agoda Services Co., Ltd. e: istvan.sz...@agoda.com

[ceph-users] Re: Libvirt and Ceph: libvirtd tries to open random RBD images

Hi, I'm not familiar with Cloudstack, I was just wondering if it tries to query the pool "rbd"? Some tools refer to a default pool "rbd" if no pool is specified. Do you have an "rbd" pool in that cluster? Another thought are namespaces, do you have those defined? Can you increase the debug

[ceph-users] Re: How to identify the index pool real usage?

Yes that's right. Test them on a single OSD, to validate. Does your platform write a lot and everywhere? From what I just saw, it seems to me that the discard only applies to transactions (and not the entire disk). If you can report back the results, that would be great. __

[ceph-users] Re: Ceph 16.2.14: osd crash, bdev() _aio_thread got r=-1 ((1) Operation not permitted)

Hi, Just to reiterate, I'm referring to an OSD crash loop because of the following error: "2023-12-03T04:00:36.686+ 7f08520e2700 -1 bdev(0x55f02a28a400 /var/lib/ceph/osd/ceph-56/block) _aio_thread got r=-1 ((1) Operation not permitted)". More relevant log entries: https://pastebin.com/gDat6rf

[ceph-users] Re: Space reclaim doesn't happening in nautilus RBD pool

Hi Istvan, The number of objects in "im" pool (918.34k) doesn't line up with "rbd du" output which says that only 2.2T are provisioned (that would take roughly ~576k objects). This usually occurs when there are object clones caused by previous snapshots -- keep in mind that trimming object clones

[ceph-users] Re: reef 18.2.1 QE Validation status

Hi Yuri, On Fri, Dec 1, 2023 at 8:47 PM Yuri Weinstein wrote: > > Venky, pls review the test results for smoke and fs after the PRs were merged. fs run looks good. Summarized here https://tracker.ceph.com/projects/cephfs/wiki/Reef#04-Dec-2023 > > Radek, Igor, Adam - any updates on http

[ceph-users] Re: MDS stuck in up:rejoin

I rebooted the servers and now the MDS won't start at all. They give the (truncated) error:     -1> 2023-12-04T14:44:41.354+ 7f6351715640 -1 ./src/mds/MDCache.cc: In function 'void MDCache::rejoin_send_rejoins()' thread 7f6351715640 time 2023-12-04T14:44:41.354292+ ./src/mds/MDCache.cc

[ceph-users] Nov/Dec Ceph Science Virtual User Group

Hey All, So I got busy and failed at getting an email out with a couple days notice for last week so let's meet up this week!  We will be having a Ceph science/research/big cluster call on Wednesday December 6th. If anyone wants to discuss something specific they can add it to the pad linked

[ceph-users] Re: Libvirt and Ceph: libvirtd tries to open random RBD images

Hello Eugen, Thanks for the response. No, we don't have a pool named "rbd" or any namespaces defined. I'll figure out a way to increase libvirtd debug level and check. Regards, Jayanth On Mon, Dec 4, 2023 at 3:16 PM Eugen Block wrote: > Hi, > > I'm not familiar with Cloudstack, I was just wonde

[ceph-users] Re: the image used size becomes 0 after export/import with snapshot

On Tue, Nov 28, 2023 at 8:18 AM Tony Liu wrote: > > Hi, > > I have an image with a snapshot and some changes after snapshot. > ``` > $ rbd du backup/f0408e1e-06b6-437b-a2b5-70e3751d0a26 > NAME > PROVISIONED USED > f04

[ceph-users] Re: [ext] CephFS pool not releasing space after data deletion

Hi Mathias/Frank, (sorry for the late reply - this didn't get much attention including the tracker report and eventually got parked). Will have this looked into - expect an update in a day or two. On Sat, Dec 2, 2023 at 5:46 PM Frank Schilder wrote: > > Hi Mathias, > > have you made any progres

[ceph-users] Re: MDS stuck in up:rejoin

Hi Eric, On Mon, Nov 27, 2023 at 8:00 PM Eric Tittley wrote: > > Hi all, > > For about a week our CephFS has experienced issues with its MDS. > > Currently the MDS is stuck in "up:rejoin" > > Issues become apparent when simple commands like "mv foo bar/" hung. I assume the MDS was active at this

[ceph-users] Re: ceph fs (meta) data inconsistent

Frank, By using your script I still couldn't reproduce it. Locally my python version is 3.9.16, and I didn't have other VMs to test python other versions. Could you check the tracker to provide the debug logs ? Thanks - Xiubo On 12/1/23 21:08, Frank Schilder wrote: Hi Xiubo, I uploaded a

[ceph-users] EC Profiles & DR

Hi All, Looking for some help/explanation around erasure code pools, etc. I set up a 3-node Ceph (Quincy) cluster with each box holding 7 OSDs (HDDs) and each box running Monitor, Manager, and iSCSI Gateway. For the record the cluster runs beautifully, without resource issues, etc. I created

[ceph-users] Re: mds slow request with “failed to authpin, subtree is being exported"

On 12/4/23 16:25, zxcs wrote: Thanks a lot, Xiubo! we already set ‘mds_bal_interval’ to 0. and the slow mds seems decrease. But somehow we still see mds complain slow request. and from mds log , can see “slow request *** seconds old, received at 2023-12-04T…: internal op exportdir:mds.* curr

[ceph-users] Re: mds slow request with “failed to authpin, subtree is being exported"

On Tue, Dec 5, 2023 at 6:34 AM Xiubo Li wrote: > > > On 12/4/23 16:25, zxcs wrote: > > Thanks a lot, Xiubo! > > > > we already set ‘mds_bal_interval’ to 0. and the slow mds seems decrease. > > > > But somehow we still see mds complain slow request. and from mds log , can > > see > > > > “slow req

[ceph-users] Re: the image used size becomes 0 after export/import with snapshot

Hi Ilya, That explains it. Thank you for clarification! Tony From: Ilya Dryomov Sent: December 4, 2023 09:40 AM To: Tony Liu Cc: ceph-users@ceph.io; d...@ceph.io Subject: Re: [ceph-users] the image used size becomes 0 after export/import with snapshot O