[ceph-users] Re: Cephadm cannot aquire lock

Hi David, It looks like we are affected by the same bug, thanks for the hint. We're running pacific 16.2.0, and I'm looking forward to upgrading to the last pacific version, but the last upgrade I tried was not successful. In hindsight, it was the same bug causing the problem. Now, my

[ceph-users] Re: mon startup problem on upgrade octopus to pacific

Hi Sebastian, On Thu, Sep 02, 2021 at 11:21:07AM +0200, Sebastian Wagner wrote: On Mon, Aug 30, 2021 at 03:52:29PM +1000, Chris Dunlop wrote: I'm stuck, mid upgrade from octopus to pacific using cephadm, at the point of upgrading the mons. Could you please verify that the mon_map of each mon

[ceph-users] Re: What's your biggest ceph cluster?

This probably provides a reasonable overview - https://ceph.io/en/news/blog/2020/public-telemetry-dashboards/, specifically the grafana dashboard is here: https://telemetry-public.ceph.com Keep in mind not all clusters have telemetry enabled The largest recorded cluster seems to be in the 32-64PB

[ceph-users] Re: power loss -> 1 osd high load for 24h

Ok will do that, I am indeed still on Nautilus, thanks Dan. > > My guess without logs is that that osd was purging PGs that had been > removed previously but not fully deleted from the disk. There have been > bugs like that fixed recently, and PG removal can be intense (unless you > run latest

[ceph-users] Re: power loss -> 1 osd high load for 24h

My guess without logs is that that osd was purging PGs that had been removed previously but not fully deleted from the disk. There have been bugs like that fixed recently, and PG removal can be intense (unless you run latest releases). Next time you have an unexplained busy osd, inject

[ceph-users] power loss -> 1 osd high load for 24h

I was told there was a power loss at the datacenter. Anyway all ceph nodes lost power, just turning them on was enough to get everything back online, no problems at all. However I had one disk/osd on a high load for day. I guess this must have been some check of ceph? How can I see this,

[ceph-users] Re: Cephadm cannot aquire lock

It may be this: https://tracker.ceph.com/issues/50526 https://github.com/alfredodeza/remoto/issues/62 Which we resolved with: https://github.com/alfredodeza/remoto/pull/63 What version of ceph are you running, and is it impacted by the above? David On Thu, Sep 2, 2021 at 9:53 AM fcid wrote:

[ceph-users] Re: ceph bluestore speed

> > I'm trying to set up a test lab with ceph (on proxmox). > I've got 3 nodes, but I figured I'd start with 1 to test out speeds and > to > learn more about the setup of ceph. I will add the 2 extra nodes later. > > One thing that was disappointing was the writing speed. > > In my setup I've

[ceph-users] ceph bluestore speed

Hi, I'm trying to set up a test lab with ceph (on proxmox). I've got 3 nodes, but I figured I'd start with 1 to test out speeds and to learn more about the setup of ceph. I will add the 2 extra nodes later. One thing that was disappointing was the writing speed. In my setup I've got 14 * 300GB

[ceph-users] Re: cephadm Pacific bootstrap hangs waiting for mon

On Thu, 2 Sept 2021 at 04:47, Sebastian Wagner wrote: > > by chance do you still have the logs of the mon the never went up? > > https://docs.ceph.com/en/latest/cephadm/troubleshooting/#checking-cephadm-logs > Not

[ceph-users] New Pacific deployment, "failed to find osd.# in keyring" errors

I'm trying to bring up a new cluster, just installed, and I'm getting errors while trying to deploy OSDs. Of the 85 candidates found, I've got 63 in and 0 up. All of the hosts were successfully added to the cluster using 'ceph orch host add ...' , but I'm seeing things in the logs like the

[ceph-users] Re: radosgw manual deployment

Hi Francesco! I wanted to point out the fact that you were checking docs from 3 different Ceph releases: master, pacific and nautilus (RH Ceph 3.x), and that probably wouldn't help to properly set up your environment. Under what kind of license would contributers make such changes? Would eg. >

[ceph-users] Re: Cephadm cannot aquire lock

Hi Sebastian, Following your sugestion, I've found this process: /usr/bin/python3 /var/lib/ceph//cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6 --image ceph/ceph@sha256:9b04c0f15704c49591640a37c7adfd40ffad0a4b42fecb950c3407687cb4f29a ceph-volume --fsid -- lvm list

[ceph-users] Is autoscale working with ec pool?

Hi, In our cluster we only the data pool is on ec 4:2, the others are on replica 3. --- RAW STORAGE --- CLASS SIZE AVAILUSED RAW USED %RAW USED nvme10 TiB 10 TiB 121 MiB 408 GiB 3.80 ssd524 TiB 440 TiB 84 TiB84 TiB 16.01 TOTAL 534 TiB 450 TiB 84

[ceph-users] pg_num number for an ec pool

Hi, For replicated pool the calculation is easy but I might be confused with replicated. I have 36 osd (15TB each), how much pg should be the data pool for objectstore located on ec 4:2? Am I correct with 512? So let's say 100pg/osd, so 36x100/6 and closest power of 2? Thank you.

[ceph-users] Re: podman daemons in error state - where to find logs?

> Really? Under what user do these containers (osd,msd,mgr etc) run then? I > have practically all containers running with specific users. Just to make > sure that if there is some sort of issue with the orchestrator, the issue > will be limited to the used userid. I think with

[ceph-users] Re: podman daemons in error state - where to find logs?

> > > a ceph user should not be required for the containers to log to systemd. > Did things end up in syslog? > Really? Under what user do these containers (osd,msd,mgr etc) run then? I have practically all containers running with specific users. Just to make sure that if there is some sort

[ceph-users] Re: mon startup problem on upgrade octopus to pacific

Could you please verify that the mon_map of each mon contains all and correct mons? Am 30.08.21 um 21:45 schrieb Chris Dunlop: Hi, Does anyone have any suggestions? Thanks, Chris On Mon, Aug 30, 2021 at 03:52:29PM +1000, Chris Dunlop wrote: Hi, I'm stuck, mid upgrade from octopus to

[ceph-users] Re: Cephadm cannot aquire lock

Am 31.08.21 um 04:05 schrieb fcid: Hi ceph community, I'm having some trouble trying to delete an OSD. I've been using cephadm in one of our clusters and it's works fine, but lately, after an OSD failure, I cannot delete it using the orchestrator. Since the orchestrator is not working (for

[ceph-users] Re: Very beginner question for cephadm: config file for bootstrap and osd_crush_chooseleaf_type

It does sets three config options things: 1. global/osd_crush_choose_leaf_type = 0 2. global/osd_pool_default_size = 2 3. mgr/mgr_standby_modules = False Am 31.08.21 um 13:08 schrieb Ignacio García: Just for experimenting, which are those single host defaults? Maybe these?:

[ceph-users] Re: cephadm Pacific bootstrap hangs waiting for mon

by chance do you still have the logs of the mon the never went up? https://docs.ceph.com/en/latest/cephadm/troubleshooting/#checking-cephadm-logs Sebastian Am 31.08.21 um 23:51 schrieb Matthew Pounsett: On Tue,

[ceph-users] Re: Brand New Cephadm Deployment, OSDs show either in/down or out/down

Can you verify that the `/usr/lib/sysctl.d/` folder exists on your debian machines? Am 01.09.21 um 15:19 schrieb Alcatraz: Sebastian, I appreciate all your help. I actually (out of desperation) spun up another cluster, same specs, just using Ubuntu 18.04 rather than Debian 10. All the OSDs

[ceph-users] Re: cephadm 15.2.14 - mixed container registries?

Am 02.09.21 um 02:54 schrieb Nigel Williams: I managed to upgrade to 15.2.14 by doing: ceph orch upgrade start --image quay.io/ceph/ceph:v15.2.14 (anything else I tried would fail) When I look in ceph orch ps output though I see quay.io for most image sources, but alertmanager, grafana,

[ceph-users] Re: podman daemons in error state - where to find logs?

We have a troubleshooting section here: https://docs.ceph.com/en/latest/cephadm/troubleshooting/#checking-cephadm-logs a ceph user should not be required for the containers to log to systemd. Did things end up

[ceph-users] [Ceph Upgrade] - Rollback Support during Upgrade failure

Hi Team, We have deployed the Ceph Octopus release using Ceph-Ansible. During the upgrade from Octopus to Pacific release we saw the upgrade got failed. We have data on OSD which we need to save. Queries : 1. How can we bring the setup back to the older normal state without impacting the

[ceph-users] Re: Replacing swift with RGW

Below are errors I am getting once I try to run swift commands . the second one is the error I get once try to install python-swiftclient (kolla-open) [stack@kolla-open ~]$ swift -v stat -bash: swift: command not found (kolla-open) [stack@kolla-open ~]$ sudo yum -y install python-swiftclient Last

[ceph-users] Re: Replacing swift with RGW

I can't tell for sure, but yes, I believe you need the openstack-swift package (with dependencies). What errors do you get? The more information you share the better people can help. Zitat von Michel Niyoyita : I tried to install "sudo yum -y install python-swiftclient" on openstack side

[ceph-users] Re: Replacing swift with RGW

I tried to install "sudo yum -y install python-swiftclient" on openstack side but fails . are there openastack-shwift packages which are needed? if are there please help me to get . may be also it is the cause I am failing to run swift command on openstack cli side. thank you for your continued

[ceph-users] Re: What's your biggest ceph cluster?

Hi, It’s not an easy question as bottleneck can be very different depending of use case/hardware. You may be interested by last ceph user survey results : https://ceph.io/en/news/blog/2021/2021-ceph-user-survey-results/

[ceph-users] What's your biggest ceph cluster?

Hi, all. I want to know that how big cluster ceph can support. Please give me some information about your ceph cluster, including cluster type say object store, file system. Thanks in advance. ___ ceph-users mailing list -- ceph-users@ceph.io To

[ceph-users] Re: Replacing swift with RGW

I only configured the endpoints for the clients to directly access the RGWs, but you'll probably need to install the openstack-swift package. Or have you done that already? Zitat von Michel Niyoyita : Thank you Eugen for your prompt response. Now the commands provided work. but I am not