[ceph-users] Re: Make Ceph available over VPN?

2022-11-07 Thread Sagittarius-A Black Hole
All the ceph nodes are part of the vpn network, so all of the nodes can be reached: in tailscale, each host gets an additional vpn ip and can be reached over tailscale from the individual client systems (laptops) when out of the office. Is there any documentation how to make ceph bind the the

[ceph-users] Re: Failed to apply 1 service(s): mon

2022-11-07 Thread Johan
Hi, As you suggested, it was the network that was wrong. It was set to 192.168.119.1/24 and when I changed it to ...119.0/24 the error went away. Even introduced the error again and the error-messages reappeared. But it is strange that I have had this error for weeks without any

[ceph-users] How to ... alertmanager and prometheus

2022-11-07 Thread Michael Lipp
Hi, I've just setup a test cluster with cephadm using quincy. Things work nicely. However, I'm not sure how to "handle" alertmanager and prometheus. Both services obviously aren't crucial to the working of the storage, fine. But there seems to be no built-in fall-over concept. By default,

[ceph-users] Re: How to manuall take down an osd

2022-11-07 Thread Anthony D'Atri
`ceph osd down` only marks the OSD down in the CRUSH map. It does not affect daemons. When OSDs are operating properly they’ll mark themselves back up. This is occasionally useful to kick stuck peering etc. The question of how to remove/replace an OSD depends on how they’re deployed. > On

[ceph-users] Re: Make Ceph available over VPN?

2022-11-07 Thread Robert Sander
Am 07.11.22 um 13:03 schrieb Sagittarius-A Black Hole: How can I add the VPN interface to Ceph to bind to for client mount requests? You cannot just terminate the VPN on one of the Ceph nodes (even when a MON is running there) and then "mount" the Ceph cluster via VPN. The Ceph client

[ceph-users] Re: ceph filesystem stuck in read only

2022-11-07 Thread Galzin Rémi
Hi Ramana and thank you, yes, before the MDS's host reboot the filesystem was read+write and the cluster was just fine too. We haven't made any upgrade since the cluster has been installed. Some times ago i had to rebuild 6 OSDs, due to start failure at boot time. No more troubles since. _ What

[ceph-users] Re: CephFS constant high write I/O to the metadata pool

2022-11-07 Thread Milind Changire
maybe, - use the top program to look at a threaded listing of the ceph-mds process and see which thread(s) are consuming the most cpu - use gstack to attach to the ceph-mds process and dump the backtrace into a file; we can then map the thread with highest cpu consumption to the

[ceph-users] Make Ceph available over VPN?

2022-11-07 Thread Sagittarius-A Black Hole
Hi, I'm using the Tailscale VPN, which is based on wireguard and want to make the Ceph cluster available offsite. Now I have seen that ceph does not bind the mon to the tailscale interface, so it is not available over it. Nmap shows the ceph port 6789 not present via Tailscale. I have two

[ceph-users] Re: CephFS constant high write I/O to the metadata pool

2022-11-07 Thread Olli Rajala
I might have spoken too soon :( Now about 60h after dropping the caches the write bandwidth has gone up linearly from those initial hundreds of kB/s to now nearly 10MB/s. I don't think this could be caused by the cache just filling up again either. After dropping the cache I tested if filling up

[ceph-users] Cephadm - db and osd partitions on same disk

2022-11-07 Thread Sean Matheny
We have a new cluster being deployed using cephadm. We have 24 18TB HDDs and 4x 2.9TB NVMs per storage node, and are wanting to use the flash drives for both rocks.db/WAL for the 24 spinners as well as flash OSDs. From first inspection it seems like cephadm only supports using a device for a

[ceph-users] Re: s3 select

2022-11-07 Thread Gal Salomon
yes. Pacific is the initial version, it includes only the CSV, and not all SQL operators Quincy: include CSV and Parquet (and almost all SQL operators) the upstream includes an initial version of JSON. I'm not sure whether it exists in Quincy. as for backporting (to older versions) I'm verifying

[ceph-users] Re: How to manuall take down an osd

2022-11-07 Thread Frank Schilder
Hi Rainer, slow ops going away after osd down sounds a little bit like this: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/E6LSKCPXPQS4G3CZUQR6M2BK5SNIZ7PX/. Read errors indicate a dying disk, however, there might be something in common here. Do you have stats of network

[ceph-users] How to manuall take down an osd

2022-11-07 Thread Rainer Krienke
Hi, today morning I had osd.77 in my ceph nautilus cluster with 144 OSDs on 9 hosts that seemed to not be working correctly , it caused slow ops: ceph -s cluster: id: 7397a0cf-bfc6-4d25-aabb-be9f6564a13b health: HEALTH_WARN Reduced data availability: 6 pgs inactive,

[ceph-users] Re: Failed to apply 1 service(s): mon

2022-11-07 Thread Eugen Block
Hi, how does your mon section of the myservice.yaml look? Could you please paste it? How did you configure the public network? Can you share # ceph config get mon public_network It sounds like you have 192.168.119.1/24 but you wanted 192.168.119.0/24 configured (no host bits set), can you