[ceph-users] NFS HA - "virtual_ip": null after upgrade to reef

Hi Last week we created an NFS service like this: " ceph nfs cluster create jumbo "ceph-flash1,ceph-flash2,ceph-flash3" --ingress --virtual_ip 172.21.15.74/22 --ingress-mode keepalive-only " Worked like a charm. Yesterday we upgraded from 17.2.7 to 18.20.0 and the NFS virtual IP seems to

[ceph-users] how to avoid pglogs dups bug in Pacific

 Hi,  I'm running Pacific 16.2.4 and I want to start a manual pg split process on the data pool (from 2048 to 4096).  I'm reluctant to upgrade to 16.2.14/15 at this point. Can I avoid the dups bug (https://tracker.ceph.com/issues/53729)  if I will increase the pgs slowly with 32 or 64pgs at

[ceph-users] Re: pacific 16.2.15 QE validation status

Update. Seeking approvals/reviews for: rados - Radek, Laura, Travis, Ernesto, Adam King rgw - Casey fs - Venky rbd - Ilya krbd - Ilya upgrade/nautilus-x (pacific) - Casey PTL (regweed tests failed) upgrade/octopus-x (pacific) - Casey PTL (regweed tests failed) upgrade/pacific-x (quincy) -

[ceph-users] Re: RGW crashes when rgw_enable_ops_log is enabled

Hi The issue is open: https://tracker.ceph.com/issues/64244 If you could take a look or let me know what are the next steps I would be super grateful. In the meantime I will try to increase the read throughput. Thanks Marc On 1/26/24 15:23, Matt Benjamin wrote: Hi Marc, 1. if you can,

[ceph-users] Re: 6 pgs not deep-scrubbed in time

actually it seems the issue I had in mind was fixed in 16.2.11 so you should be fine. Respectfully, *Wes Dillingham* w...@wesdillingham.com LinkedIn On Tue, Jan 30, 2024 at 10:34 AM Wesley Dillingham wrote: > You may want to consider upgrading to

[ceph-users] Re: 6 pgs not deep-scrubbed in time

You may want to consider upgrading to 16.2.14 before you do the pg split. Respectfully, *Wes Dillingham* w...@wesdillingham.com LinkedIn On Tue, Jan 30, 2024 at 10:18 AM Michel Niyoyita wrote: > I tried that on one of my pool (pool id 3) but the

[ceph-users] Re: 6 pgs not deep-scrubbed in time

I tried that on one of my pool (pool id 3) but the number of pgs not deep-scrubbed in time increased also from 55 to 100 but the number of PGs was increased. I set also autoscale to off mode. before continue to other pools would like to ask if so far there is no negative impact. ceph -s

[ceph-users] Re: 6 pgs not deep-scrubbed in time

It will take a couple weeks to a couple months to complete is my best guess on 10TB spinners at ~40% full. The cluster should be usable throughout the process. Keep in mind, you should disable the pg autoscaler on any pool which you are manually adjusting the pg_num for. Increasing the pg_num is

[ceph-users] Re: 6 pgs not deep-scrubbed in time

Thanks for your advices Wes, below is what ceph osd df tree shows , the increase of pg_num of the production cluster will not affect the performance or crush ? how long it can takes to finish? ceph osd df tree ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL

[ceph-users] Re: 6 pgs not deep-scrubbed in time

I now concur you should increase the pg_num as a first step for this cluster. Disable the pg autoscaler for and increase the volumes pool to pg_num 256. Then likely re-asses and make the next power of 2 jump to 512 and probably beyond. Keep in mind this is not going to fix your short term

[ceph-users] Ceph stretch mode connect to local datacenter

Hello. I have a ceph cluster which works in stretch mode: *DC1:* node1 (osd, mon, mgr) node2 (osd, mon) node3 (osd, mds) *DC2:* node1 (osd, mon, mgr) node2 (osd, mon) node3 (osd, mds) *DC3:* node1 (mon) Datacenters are distributions between different locations. I use RBD on my clients. How

[ceph-users] Re: Scrubbing?

Ah, yeah, you hit https://tracker.ceph.com/issues/63389 during the upgrade. Josh On Tue, Jan 30, 2024 at 3:17 AM Jan Marek wrote: > > Hello again, > > I'm sorry, I forgot attach file... :-( > > Sincerely > Jan > > Dne Út, led 30, 2024 at 11:09:44 CET napsal(a) Jan Marek: > > Hello Sridhar, > >

[ceph-users] Re: pacific 16.2.15 QE validation status

dashboard looks good! approved. Regards, Nizam On Tue, Jan 30, 2024 at 3:09 AM Yuri Weinstein wrote: > Details of this release are summarized here: > > https://tracker.ceph.com/issues/64151#note-1 > > Seeking approvals/reviews for: > > rados - Radek, Laura, Travis, Ernesto, Adam King > rgw -

[ceph-users] Re: Scrubbing?

Hello again, I'm sorry, I forgot attach file... :-( Sincerely Jan Dne Út, led 30, 2024 at 11:09:44 CET napsal(a) Jan Marek: > Hello Sridhar, > > at Saturday I've finished upgrade proces to 18.2.1. > > Cluster is now in HEALTH_OK state and performs well. > > According to my colleagues there

[ceph-users] Re: Scrubbing?

Hello Sridhar, at Saturday I've finished upgrade proces to 18.2.1. Cluster is now in HEALTH_OK state and performs well. According to my colleagues there are lower latences and good throughput. On OSD nodes there is relative low I/O activity. I still have mClock profile "high_client_ops".

[ceph-users] Re: pacific 16.2.15 QE validation status

Hi Yuri, The ceph-volume failure is a valid bug. Investigating for the root cause of it and will submit a patch. Thanks! -- Guillaume Abrioux Software Engineer From: Yuri Weinstein Date: Monday, 29 January 2024 at 22:38 To: dev , ceph-users Subject: [EXTERNAL] [ceph-users] pacific 16.2.15 QE

[ceph-users] cephfs inode backtrace information

Hello, I have a question regarding the default pool of a cephfs. According to the docs it is recommended to use a fast ssd replicated pool as default pool for cephfs. I'm asking what are the space requirements for storing the inode backtrace information? Let's say I have a 85 TiB replicated

[ceph-users] Changing A Ceph Cluster's Front- And/Or Back-End Networks IP Address(es)

Hi All, Quick Q: How easy/hard is it to change the IP networks of: 1) A Ceph Cluster's "Front-End" Network? 2) A Ceph Cluster's "Back-End" Network? Is it a "simply" matter of: a) Placing the Nodes in maintenance mode b) Changing a config file (I assume it's /etc/ceph/ceph.conf) on each Node

[ceph-users] Re: 6 pgs not deep-scrubbed in time

Dear team, below is the output of ceph df command and the ceph version I am running ceph df --- RAW STORAGE --- CLASS SIZEAVAIL USED RAW USED %RAW USED hdd433 TiB 282 TiB 151 TiB 151 TiB 34.82 TOTAL 433 TiB 282 TiB 151 TiB 151 TiB 34.82 --- POOLS --- POOL