[ceph-users] Recommended procedure to modify Crush Map

Hi, my current Crush Map includes multiple roots representing different disks. There are multiple crush rules, one for each pool. And a pool represents a disk type: hdd, ssd, nvme Question: What is the recommended procedure to modify the Crush Map in order to define only one root and "transfer" a

[ceph-users] Starting service rbd-target-api fails

Hi, I want to setup Ceph iSCSI Gateway and I follow this documentation. In step "Setup" of process "Configuring the iSCSI target using the command line interface " I cannot start servic

[ceph-users] Re: mds crash loop

Thank you Yan, Zheng for the help to get my cephfs back in working order by providing a source version that had the fix in it to get the root inode fixed. (https://tracker.ceph.com/issues/42675) - Karsten -Original message- From: Yan, Zheng Sent: Tue 12-11-2019 11:55 Subject:

[ceph-users] Re: bluestore rocksdb behavior

Unfortunately can't recall any On 12/4/2019 11:07 PM, Frank R wrote: Thanks. Can you recommend any docs for understanding the BlueStore on disk format/behavior when there is no separate device for the WAL/DB? On Wed, Dec 4, 2019 at 10:19 AM Igor Fedotov > wrote:

[ceph-users] Re: Starting service rbd-target-api fails

On 12/05/2019 03:16 AM, Thomas Schneider wrote: > Hi, > > I want to setup Ceph iSCSI Gateway and I follow this > documentation. > In step "Setup" of process "Configuring the iSCSI target using the > command line interface >

[ceph-users] Re: [Ceph-users] Re: MDS failing under load with large cache sizes

I had similar issues again today. Some users were trying to train a neural network on several million files resulting in enormous cache sizes. Due to my custom cap recall and decay rate settings, the MDSs were able to withstand the load for quite some time, but at some point the active rank crashed

[ceph-users] Re: [Ceph-users] Re: MDS failing under load with large cache sizes

On Thu, Dec 5, 2019 at 10:31 AM Janek Bevendorff wrote: > > I had similar issues again today. Some users were trying to train a > neural network on several million files resulting in enormous cache > sizes. Due to my custom cap recall and decay rate settings, the MDSs > were able to withstand the

[ceph-users] 2 different ceph-users lists?

Hi, Are there 2 different ceph-users list? ceph-us...@lists.ceph.com and ceph-users@ceph.io Why? What's the difference? Regards, Rodrigo Severo ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@cep

[ceph-users] Re: 2 different ceph-users lists?

ceph-us...@lists.ceph.com is old one, why this is, I also do not know https://www.mail-archive.com/search?l=all&q=ceph -Original Message- From: Rodrigo Severo - Fábrica [mailto:rodr...@fabricadeideias.com] Sent: donderdag 5 december 2019 20:37 To: ceph-us...@lists.ceph.com; ceph-user

[ceph-users] Re: 2 different ceph-users lists?

Em qui., 5 de dez. de 2019 às 16:38, Marc Roos escreveu: > > > > ceph-us...@lists.ceph.com is old one, why this is, I also do not know Ok Marc. Thanks for your information. Rodrigo ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send

[ceph-users] Crushmap format in nautilus: documentation out of date

Hi, The crushmap produced by ceph osd getcrushmap in ceph version 14.2.4 has more info than defined in https://docs.ceph.com/docs/cuttlefish/rados/operations/crush-map/ There is a second id per bucket: host a1-df { id -3 # do not change unnecessarily id -4 class hdd # do not change u

[ceph-users] Re: [Ceph-users] Re: MDS failing under load with large cache sizes

> You set mds_beacon_grace ? Yes, as I said. It seemed to have no effect or at least none that I could see. The kick timeout seemed random after all. I even set it to something ridiculous like 1800 and the MDS were still timed out. Sometimes they got to 20M inodes, sometimes only to a few 100k.

[ceph-users] Re: iSCSI Gateway reboots and permanent loss

On 12/04/2019 02:34 PM, Wesley Dillingham wrote: > I have never had a permanent loss of a gateway but I'm a believer in > Murphy's law and want to have a plan. Glad to hear that there is a > solution in-the-works, curious when might that be available in a > release? If sooner than later I'll plan t

[ceph-users] Re: iSCSI Gateway reboots and permanent loss

Thats great thank you so much. I will try and get this patch in my test env asap but will likely wait for official release cut for prod. I really appreciate you adding this in to the product. Respectfully, *Wes Dillingham* w...@wesdillingham.com LinkedIn

[ceph-users] Upgrade from Jewel to Nautilus

Hello, We are planning to upgrade our cluster from Jewel to Nautilus. From my understanding, leveldb of monitor and filestore of OSDs will not be converted to rocketdb and bluestore automatically. So do you suggest to convert them manually after upgrading software? Is there any document or guid

[ceph-users] Re: Upgrade from Jewel to Nautilus

You should definitely migrate to BlueStore, that'll also take care of the leveldb/rocksdb upgrade :) For me mons: as it's super easy to delete and re-create a mon that's usually the best/simplest way to go. Also, note that you can't immediately continue from Luminous to Nautilus, you have to scrub

[ceph-users] Re: bluestore rocksdb behavior

There's some good information here which may assist in your understanding. https://www.youtube.com/channel/UCno-Fry25FJ7B4RycCxOtfw/search?query=bluestore On Thu, Dec 5, 2019 at 10:36 PM Igor Fedotov wrote: > > Unfortunately can't recall any > > On 12/4/2019 11:07 PM, Frank R wrote: > > Thanks.

[ceph-users] Re: Crushmap format in nautilus: documentation out of date

The second id is to overlay device types in the crush map. In the olden days, we would define two seperate roots, one for hdd and another for SSD (each host would have a duplicate entry with a slightly different name and different id), then have a crush rule use the different root for the type of s

[ceph-users] Re: Starting service rbd-target-api fails

Hi Mike, actually you point to the right log; I can find relevant information in this logfile /var/log/rbd-target-api/rbd-target-api.log: root@ld5505:~# tail -f /var/log/rbd-target-api/rbd-target-api.log 2019-12-04 12:09:52,986    ERROR [rbd-target-api:2918:()] - 'rbd' pool does not exist! 2019-12

[ceph-users] High swap usage on one replication node

Hi, our Ceph 14.2.3 cluster so far runs smooth with replicated and EC pools, but since a couple of days one of the dedicated replication nodes consumes up to 99% swap and stays at that level. The other two replicated nodes use +- 50 - 60% of swap. All the 24 NVMe OSDs per node are BlueStore wi