[ceph-users] Re: Correct Usage of the ceph-objectstore-tool??

[Lee]

Cheers for the explanation..

I found the problem, Ceph had marked the new OSD as Out, as soon as I
marked it in, then it started to rebuild.

Thankyou for taking the time to explain the above.

Looks like i have everything back (fingers crossed).

On Fri, 7 Jan 2022 at 03:08, Alexander E. Patrakov 
wrote:

> пт, 7 янв. 2022 г. в 06:21, Lee :
>
>> Hello,
>>
>> As per another post I been having a huge issue since a PGNUM increase took
>> my cluster offline..
>>
>> I have got to a point where I have just 20 PG's Down / Unavailable due to
>> not being able to start a OSD
>>
>> I have been able to export the PG from the the offline OSD
>>
>> I then import to a clean / new OSD which is set to weight 0 in Crush using
>> the following command.
>>
>> ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-17
>> --no-mon-config
>> --op import --file /mnt/croit/backup/pg10.0
>>
>
> If you did this with the OSD being offline, and then started it, then you
> did everything correctly. OTOH my preferred approach would be not to set
> the weight to 0, but to create a separate otherwise-unused CRUSH pool and
> assign the OSDs with extra data to it, but your approach is also valid.
>
>
>> When I start the OSD I see in the log loads of stuff being transitioned to
>> Stray.
>>
>
> This is an indicator that you did everything right. Stray means an extra
> copy of data in a place where it is not supposed to be - but that's exactly
> what you did and what you were supposed to do!
>
>
>> Do I need to tell CEPH to used the pg on the OSD to rebuild? When I query
>> the PG at the end it complains about marking the offline OSD as offline?
>>
>
> We need to understand why this happens. The usual scenario where the
> automatic rebuild doesn't start is when some of the PGs that you exported
> and imported do not represent the known latest copy of the data. Maybe
> there is another copy on another dead OSD, try exporting and importing it,
> too. Basically, you want to inject all copies of all PGs that are
> unhealthy. A copy of "ceph pg dump" output (as an attached file, not
> inline) might be helpful. Also, run "ceph pg 1.456 query" where 1.456 is
> the PG ID that you have imported - for a few problematic PGs.
>
>
>> I have looked online and cannot find a definitive guide on how the process
>> / steps that should be taken.
>>
>
> There is no single definitive guide. My approach would be to treat the
> broken OSDs as broken for good, but without using any command that includes
> the word "lost" (because this actually loses data). You can mark the dead
> OSDs as "out", if Ceph didn't do it for you already. Then add enough
> capacity with non-zero weight in the correct pool (or maybe do nothing if
> you already have enough space). Ceph will rebalance the data automatically
> when it obtains a proof that it really has the latest data.
>
> When I encountered this "dead osd" issue last time, I found it useful to
> compare the "ceph health detail" output over time, and with/without the
> OSDs with injected PGs running. At the very least, it provides a useful
> metric of what is remaining to do.
>
> Also an interesting read-only command (but maybe for later) would be:
> "ceph osd safe-to-destroy 123" where 123 is the dead OSD id.
>
> --
> Alexander E. Patrakov
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Correct Usage of the ceph-objectstore-tool??

[Alexander E. Patrakov]

пт, 7 янв. 2022 г. в 06:21, Lee :

> Hello,
>
> As per another post I been having a huge issue since a PGNUM increase took
> my cluster offline..
>
> I have got to a point where I have just 20 PG's Down / Unavailable due to
> not being able to start a OSD
>
> I have been able to export the PG from the the offline OSD
>
> I then import to a clean / new OSD which is set to weight 0 in Crush using
> the following command.
>
> ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-17 --no-mon-config
> --op import --file /mnt/croit/backup/pg10.0
>

If you did this with the OSD being offline, and then started it, then you
did everything correctly. OTOH my preferred approach would be not to set
the weight to 0, but to create a separate otherwise-unused CRUSH pool and
assign the OSDs with extra data to it, but your approach is also valid.


> When I start the OSD I see in the log loads of stuff being transitioned to
> Stray.
>

This is an indicator that you did everything right. Stray means an extra
copy of data in a place where it is not supposed to be - but that's exactly
what you did and what you were supposed to do!


> Do I need to tell CEPH to used the pg on the OSD to rebuild? When I query
> the PG at the end it complains about marking the offline OSD as offline?
>

We need to understand why this happens. The usual scenario where the
automatic rebuild doesn't start is when some of the PGs that you exported
and imported do not represent the known latest copy of the data. Maybe
there is another copy on another dead OSD, try exporting and importing it,
too. Basically, you want to inject all copies of all PGs that are
unhealthy. A copy of "ceph pg dump" output (as an attached file, not
inline) might be helpful. Also, run "ceph pg 1.456 query" where 1.456 is
the PG ID that you have imported - for a few problematic PGs.


> I have looked online and cannot find a definitive guide on how the process
> / steps that should be taken.
>

There is no single definitive guide. My approach would be to treat the
broken OSDs as broken for good, but without using any command that includes
the word "lost" (because this actually loses data). You can mark the dead
OSDs as "out", if Ceph didn't do it for you already. Then add enough
capacity with non-zero weight in the correct pool (or maybe do nothing if
you already have enough space). Ceph will rebalance the data automatically
when it obtains a proof that it really has the latest data.

When I encountered this "dead osd" issue last time, I found it useful to
compare the "ceph health detail" output over time, and with/without the
OSDs with injected PGs running. At the very least, it provides a useful
metric of what is remaining to do.

Also an interesting read-only command (but maybe for later) would be: "ceph
osd safe-to-destroy 123" where 123 is the dead OSD id.

-- 
Alexander E. Patrakov
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Correct Usage of the ceph-objectstore-tool??

[Lee]

Hello,

As per another post I been having a huge issue since a PGNUM increase took
my cluster offline..

I have got to a point where I have just 20 PG's Down / Unavailable due to
not being able to start a OSD

I have been able to export the PG from the the offline OSD

I then import to a clean / new OSD which is set to weight 0 in Crush using
the following command.

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-17 --no-mon-config
--op import --file /mnt/croit/backup/pg10.0

When I start the OSD I see in the log loads of stuff being transitioned to
Stray.

Do I need to tell CEPH to used the pg on the OSD to rebuild? When I query
the PG at the end it complains about marking the offline OSD as offline?

I have looked online and cannot find a definitive guide on how the process
/ steps that should be taken.

Cheers

Lee
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Help - Multiple OSD's Down

[Lee]

I tried with disk based swap on a SATA SSD.

I think that might be the last option. I have exported already all the down
PG's from the OSD that they are waiting for.

Kind Regards

Lee

On Thu, 6 Jan 2022 at 20:00, Alexander E. Patrakov 
wrote:

> пт, 7 янв. 2022 г. в 00:50, Alexander E. Patrakov :
>
>> чт, 6 янв. 2022 г. в 12:21, Lee :
>>
>>> I've tried add a swap and that fails also.
>>>
>>
>> How exactly did it fail? Did you put it on some disk, or in zram?
>>
>> In the past I had to help a customer who hit memory over-use when
>> upgrading Ceph (due to shallow_fsck), and we were able to fix it by adding
>> 64 GB GB of zram-based swap on each server (with 128 GB of physical RAM in
>> this type of server).
>>
>>
> On the other hand, if you have some spare disks for temporary storage and
> for new OSDs, and this failed OSD is not a part of an erasure-coded pool,
> another approach might be to export all PGs using ceph-objectstore-tool as
> files onto the temporary storage (in hope that it doesn't suffer from the
> same memory explosion), and then import them all into a new temporary OSD.
>
> --
> Alexander E. Patrakov
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Help - Multiple OSD's Down

[Alexander E. Patrakov]

пт, 7 янв. 2022 г. в 00:50, Alexander E. Patrakov :

> чт, 6 янв. 2022 г. в 12:21, Lee :
>
>> I've tried add a swap and that fails also.
>>
>
> How exactly did it fail? Did you put it on some disk, or in zram?
>
> In the past I had to help a customer who hit memory over-use when
> upgrading Ceph (due to shallow_fsck), and we were able to fix it by adding
> 64 GB GB of zram-based swap on each server (with 128 GB of physical RAM in
> this type of server).
>
>
On the other hand, if you have some spare disks for temporary storage and
for new OSDs, and this failed OSD is not a part of an erasure-coded pool,
another approach might be to export all PGs using ceph-objectstore-tool as
files onto the temporary storage (in hope that it doesn't suffer from the
same memory explosion), and then import them all into a new temporary OSD.

-- 
Alexander E. Patrakov
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Help - Multiple OSD's Down

[Alexander E. Patrakov]

чт, 6 янв. 2022 г. в 12:21, Lee :

> I've tried add a swap and that fails also.
>

How exactly did it fail? Did you put it on some disk, or in zram?

In the past I had to help a customer who hit memory over-use when upgrading
Ceph (due to shallow_fsck), and we were able to fix it by adding 64 GB GB
of zram-based swap on each server (with 128 GB of physical RAM in this type
of server).

-- 
Alexander E. Patrakov
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] switching ceph-ansible from /dev/sd to /dev/disk/by-path

[Dave Holland]

At some point, possibly due to a BIOS/firmware or kernel/OS update,
the device naming on our SuperMicro Ceph servers has stopped being
consistent after a reboot. e.g. the internal (OS) disks might be
/dev/sda and sdb, or they might be /dev/sdbi and sdbj. This doesn't
affect Ceph's day-to-day running, but it does break ceph-ansible
because that tries to activate an OSD on the OS disk, which fails
(safely).

The right solution appears to be to configure ceph-ansible to use
/dev/disk/by-path device names, allowing for the expander IDs being
embedded in the device name -- so those would have to be set per-host
with host vars. Has anyone done that change from /dev/sd and
/dev/disk/by-path and have any advice, please? Is it a safe change, or
do I have to stick with /dev/sd names and modify the device list as a
host var, if/when the naming changes after a reboot? (Which would be
grotty!)

thanks,
Dave
-- 
**   Dave Holland   ** Systems Support -- Informatics Systems Group **
** d...@sanger.ac.uk **Wellcome Sanger Institute, Hinxton, UK**


-- 
 The Wellcome Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Infinite Dashboard 404 Loop On Failed SAML Authentication

[Edward R Huyer]

Ok, I think I've nearly got the dashboard working with SAML/Shibboleth 
authentication, except for one thing:  If a user authenticates via SAML, but a 
corresponding dashboard user hasn't been created, it triggers a loop where the 
browser gets redirected to a nonexistent dashboard unauthorized page, then to a 
nonexistent dashboard 404 page, then back to the Shibboleth authentication page 
(which succeeds instantly), then back to the unauthorized page, etc.

On a fast machine and network, this loops very quickly and thrashes the 
authentication server.  I haven't found a way to fix it.  It looks sort of like 
a bug to me?

Any suggestions?

-
Edward Huyer
Golisano College of Computing and Information Sciences
Rochester Institute of Technology
Golisano 70-2373
152 Lomb Memorial Drive
Rochester, NY 14623
585-475-6651
erh...@rit.edu

Obligatory Legalese:
The information transmitted, including attachments, is intended only for the 
person(s) or entity to which it is addressed and may contain confidential 
and/or privileged material. Any review, retransmission, dissemination or other 
use of, or taking of any action in reliance upon this information by persons or 
entities other than the intended recipient is prohibited. If you received this 
in error, please contact the sender and destroy any copies of this information.

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: fs rename: invalid command

[Aaron Oneal]

Thank you, looking forward to that. Any interim workarounds? Roadmap makes me 
think Quincy might not be available for a while and I was hoping to avoid a 
bunch of dependencies on the current FS name in use.

> On Jan 4, 2022, at 8:16 AM, Patrick Donnelly  wrote:
> 
> "ceph fs rename" will be available in Quincy. see also:
> https://tracker.ceph.com/issues/47276
> 
> On Sat, Jan 1, 2022 at 10:26 PM Aaron Oneal  wrote:
>> 
>> Hi all, I’m trying to rename a file system with the “fs rename” command and 
>> it seems to not be supported by my 16.2.7 Debian client. Any recommendations?
>> 
>> no valid command found; 4 closest matches:
>> fs rm  [--yes-i-really-mean-it]
>> fs reset  [--yes-i-really-mean-it]
>> fs required_client_features  add|rm 
>> fs rm_data_pool  
>> Error EINVAL: invalid command
>> ___
>> ceph-users mailing list -- ceph-users@ceph.io
>> To unsubscribe send an email to ceph-users-le...@ceph.io
> 
> 
> 
> -- 
> Patrick Donnelly, Ph.D.
> He / Him / His
> Principal Software Engineer
> Red Hat, Inc.
> GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
> 

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Repair/Rebalance slows down

[Ray Cunningham]

I can do that for adding new drives, but what about when removing failed disks? 
There’s not one OSD to run an iostat on? The performance profile is the same 
for adds and removes.

Thank you,
Ray

From: David Orman 
Sent: Thursday, January 6, 2022 9:18 AM
To: Ray Cunningham 
Cc: ceph-users@ceph.io
Subject: Re: [ceph-users] Repair/Rebalance slows down

What's iostat show for the drive in question? What you're seeing is the cluster 
rebalancing initially, then at the end, it's probably that single drive being 
filled. I'd expect 25-100MB/s to be the fill rate of the newly added drive with 
backfills per osd set to 2 or so (much more than that doesn't help). Check the 
disk utilization for the newly added OSD at the tail end, and you'll probably 
see it IOPS saturated.

On Thu, Jan 6, 2022 at 8:09 AM Ray Cunningham 
mailto:ray.cunning...@keepertech.com>> wrote:
Hi Everyone!

I have a 16 node, 640 OSD (5 to 1 SSD) bluestore cluster which is mainly used 
for RGW services. It has its own backend cluster network for IO separate from 
the customer network.

Whenever we add or remove an OSD the rebalance or repair IO starts off very 
fast 4GB/s+ but it will continually slow down over a week and by then end it's 
moving at KB/s. So each 16TB OSD takes a week+ to repair or rebalance! I have 
not been able to identify any bottleneck or slow point, it just seems to be 
Ceph taking longer to do its thing.

Are there any settings I can check or change to get the repair speed to 
maintain a high level to completion? If we could stay in the GB/s speed we 
should be able to repair in a couple days, not a week or more...

Thank you,
Ray

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to 
ceph-users-le...@ceph.io
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Repair/Rebalance slows down

[David Orman]

What's iostat show for the drive in question? What you're seeing is the
cluster rebalancing initially, then at the end, it's probably that single
drive being filled. I'd expect 25-100MB/s to be the fill rate of the newly
added drive with backfills per osd set to 2 or so (much more than that
doesn't help). Check the disk utilization for the newly added OSD at the
tail end, and you'll probably see it IOPS saturated.

On Thu, Jan 6, 2022 at 8:09 AM Ray Cunningham 
wrote:

> Hi Everyone!
>
> I have a 16 node, 640 OSD (5 to 1 SSD) bluestore cluster which is mainly
> used for RGW services. It has its own backend cluster network for IO
> separate from the customer network.
>
> Whenever we add or remove an OSD the rebalance or repair IO starts off
> very fast 4GB/s+ but it will continually slow down over a week and by then
> end it's moving at KB/s. So each 16TB OSD takes a week+ to repair or
> rebalance! I have not been able to identify any bottleneck or slow point,
> it just seems to be Ceph taking longer to do its thing.
>
> Are there any settings I can check or change to get the repair speed to
> maintain a high level to completion? If we could stay in the GB/s speed we
> should be able to repair in a couple days, not a week or more...
>
> Thank you,
> Ray
>
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Un-unprotectable snapshot

[Flemming Frandsen]

I have a problem with a snapshot that I cannot unprotect, it claims to be
in use and it says it has a child image, but that image has been flattened
and no longer has a parent:

root@ardmore:~# rbd snap unprotect
kube/kubernetes-dynamic-pvc-85ebad61-5a3e-11eb-b586-6af35a867cdb@crucible-crucible-data-crucible-0
2022-01-06 13:51:57.482 7fe5cdffb700 -1 librbd::SnapshotUnprotectRequest:
cannot unprotect: at least 1 child(ren) [b9e9336b8b4567] in pool 'kube'
2022-01-06 13:51:57.486 7fe5cdffb700 -1 librbd::SnapshotUnprotectRequest:
encountered error: (16) Device or resource busy
2022-01-06 13:51:57.486 7fe5cdffb700 -1 librbd::SnapshotUnprotectRequest:
0x560af83e35e0 should_complete_error: ret_val=-16
2022-01-06 13:51:57.486 7fe5cdffb700 -1 librbd::SnapshotUnprotectRequest:
0x560af83e35e0 should_complete_error: ret_val=-16
rbd: unprotecting snap failed: (16) Device or resource busy
root@ardmore:~# rbd children
kube/kubernetes-dynamic-pvc-85ebad61-5a3e-11eb-b586-6af35a867cdb@crucible-crucible-data-crucible-0

kube/csi-vol-crucible-data-crucible-0-9794703bb3b
root@ardmore:~# rbd flatten
kube/csi-vol-crucible-data-crucible-0-9794703bb3b

Image flatten: 0% complete...failed.
rbd: flatten error: (22) Invalid argument
2022-01-06 13:52:08.746 7f093568b0c0 -1 librbd::Operations: image has no
parent

I'm running Nautilus.

Any ideas?

-- 
Flemming Frandsen - YAPH - http://osaa.dk - http://dren.dk/
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Repair/Rebalance slows down

[Ray Cunningham]

Hi Everyone!

I have a 16 node, 640 OSD (5 to 1 SSD) bluestore cluster which is mainly used 
for RGW services. It has its own backend cluster network for IO separate from 
the customer network.

Whenever we add or remove an OSD the rebalance or repair IO starts off very 
fast 4GB/s+ but it will continually slow down over a week and by then end it's 
moving at KB/s. So each 16TB OSD takes a week+ to repair or rebalance! I have 
not been able to identify any bottleneck or slow point, it just seems to be 
Ceph taking longer to do its thing.

Are there any settings I can check or change to get the repair speed to 
maintain a high level to completion? If we could stay in the GB/s speed we 
should be able to repair in a couple days, not a week or more...

Thank you,
Ray

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Question about multi-site sync policies

[Ulrich Klein]

Hi,

My first question on this list …. 2nd attempt because the first one didn’t make 
it (I hope)?

I'm trying out RGW multi-site sync policies.

I have a test/PoC setup with 16.2.6 using cephadm (which, by the way, I DO like)
I only use RGW/S3 
There is one realm (myrealm), one zonegroup (myzg) and 3 zones: Zone A, zone B 
and zone DR

Zone A and B are on the same ceph instance with different rgw processes and 
pools, zone DR is on a second cluster, in case that matters.
Without any sync policies all data is synced perfectly fine between all zones

What I want to see is:
Data from zone A is synced directionally to zone DR, but not to zone B
Data from zone B is not synced anywhere
Data from zone DR is not synced back/anywhere
i.e. clients writing to zone A get their data "backed up" to zone DR, 
while clients writng to zone B don't get their data "backed" up to a second 
zone.
Clients don't have access to zone DR.

I used sync policies this way:
radosgw-admin sync group create --group-id=drsync --status=allowed
radosgw-admin sync group flow create --group-id=drsync --flow-id=a2dr 
--flow-type=directional --source-zone=a --dest-zone=dr
radosgw-admin sync group pipe create --group-id=drsync --pipe-id=allbuck 
--source-zones='*' --source-bucket='*' --dest-zones='*' --dest-bucket='*'
radosgw-admin sync group modify --group-id=drsync --status=enabled
radosgw-admin period update --commit

Now, from a data-in-buckets perspective all looks fine
Add data to bucket buck on A -> appears on buck on DR, but not in B
Add data to bucket buck on B -> doesn't appear in A or DR
Same for all other combinations, just as I want it.

BUT
radosgw-admin sync status 
with --rgw-zone=A or DR or B
after adding or removing data always shows some
"data is behind on X shards", apparently for all the shards that are - 
intentionally - not synced. Those “behind” shards accumulate over time and 
never go away again.

Is that just annoying but normal? Or is that a bug?
Or is my configuration just "bad" and could be changed so I don't have those 
sort-of errors with the sync status.

When I used bucket level sync policies (only sync certain buckets from zone A 
to DR, no zone B) I had iirc the same effect.

What I'm really trying to achieve is something like "user sync policies", i.e.
- User X data should be synced from A to DR
- User Y data should only stay in A
I’m trying to emulate that by using the existing/documented sync policies. User 
X gets URL for zone A, user Y gets URL for zone B
(or with bucket level policies both get URL for zone A and for user X syncing 
is turned on “on demand” for certain existing buckets - inconvenient)

Best would be if I could flip a flag and change the behaviour per user :)
And even better if it was easy to understand, i.e. if user's sync-flag is 
turned on all his data is synced, not just new data. If user's sync-flag is 
turned off, all his data is removed from the DR zone.


Thanks for any input :)

Ciao, Uli
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Help - Multiple OSD's Down

[Marc]

> I assume the huge memory consumption is temporary. Once the OSD is up and
> stable, it would release the memory.
> 
> So how about allocate a large swap temporarily just to let the OSD up. I
> remember that someone else on the list have resolved a similar issue with
> swap.

But is this already a reported bug, or should I from now on take into account 
that osd's can consume >150GB of memory
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Help - Multiple OSD's Down

[Marc]

Running your osd's with resource limitations is not so straightforward. I can 
guess that if you are running close to full resource utilization on your nodes, 
it makes more sense to make sure everything stays as much within their 
specified limits. (Aside from the question if you would even want to operate 
such environment, and aside from the question if you even want to force osd's 
into oom)

However if you are not walking such thin line, and have eg. more memory 
available. It is just not good, not using that memory. I do not really know how 
advanced most orchestrators are nowadays, and if you can dynamically change 
resource limits on containers. But if not, you will just not use memory as 
cache, and not using memory as cache means increased disk io, and decreased 
performance.

I think the linux kernel is probably better in deciding how to share resources 
among my osd's than I am. And that is a reason why I do not put them in 
containers. ( but I am still on nautilus, so I will keep an eye on this 'noisy' 
when upgrading ;) )


> 
> and that is exactly why I run osds containerized with limited cpu and
> memory as well as "bluestore cache size", "osd memory target", and "mds
> cache memory limit".  Osd processes have become noisy neighbors in the
> last
> few versions.
> 

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io