[ceph-users] Re: Cephadm orchestrator and special label _admin in 17.2.7
On 23.01.2024 18:19, Albert Shih wrote: Just like to known if it's a very bad idea to do a rsync of /etc/ceph from the «_admin» server to the other ceph cluster server. I in fact add something like for host in `cat /usr/local/etc/ceph_list_noeuds.txt` do /usr/bin/rsync -av /etc/ceph/ceph* $host:/etc/ceph/ done in a cronjob Why not just add the _admin label to the host and let Ceph do the job? You can also run this to get the ceph.conf copied to all host ceph config set mgr/cephadm/manage_etc_ceph_ceph_conf true Anyway, I don't se any problem with rsync it, it's just ceph.conf and the admin key. -- Kai Stian Olstad ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Cephadm orchestrator and special label _admin in 17.2.7
Le 18/01/2024 à 11:58:47+0100, Robert Sander a écrit Hi, > > According to the documentation¹ the special host label _admin instructs the > cephadm orchestrator to place a valid ceph.conf and the > ceph.client.admin.keyring into /etc/ceph of the host. > > I noticed that (at least) on 17.2.7 only the keyring file is placed in > /etc/ceph, but not ceph.conf. > > Both files are placed into the /var/lib/ceph//config directory. > > Has something changed? > Just like to known if it's a very bad idea to do a rsync of /etc/ceph from the «_admin» server to the other ceph cluster server. I in fact add something like for host in `cat /usr/local/etc/ceph_list_noeuds.txt` do /usr/bin/rsync -av /etc/ceph/ceph* $host:/etc/ceph/ done in a cronjob Regards. -- Albert SHIH 嶺 France Heure locale/Local time: mar. 23 janv. 2024 18:18:39 CET ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Cephadm orchestrator and special label _admin in 17.2.7
Oh that does sound strange indeed. I don't have a good idea right now, hopefully someone from the dev team can shed some light on this. Zitat von Robert Sander : Hi, more strang behaviour: When I isssue "ceph mgr fail" a backup MGR takes over and updates all config files on all hosts including /etc/ceph/ceph.conf. At first I thought that this was the solution but when I now remove the _admin label and add it again the new MGR also does not update /etc/ceph/ceph.conf. Only when I again do "ceph mgr fail" the new MGR will update /etc/ceph/ceph.conf on the hosts labeled with _admin. Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Cephadm orchestrator and special label _admin in 17.2.7
Hi, more strang behaviour: When I isssue "ceph mgr fail" a backup MGR takes over and updates all config files on all hosts including /etc/ceph/ceph.conf. At first I thought that this was the solution but when I now remove the _admin label and add it again the new MGR also does not update /etc/ceph/ceph.conf. Only when I again do "ceph mgr fail" the new MGR will update /etc/ceph/ceph.conf on the hosts labeled with _admin. Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Cephadm orchestrator and special label _admin in 17.2.7
On 1/18/24 14:28, Eugen Block wrote: Is your admin keyring under management? There is no issue with the admin keyring but with ceph.conf. The config setting "mgr mgr/cephadm/manage_etc_ceph_ceph_conf" is set to true and "mgr mgr/cephadm/manage_etc_ceph_ceph_conf_hosts" was at "*", so all hosts. I have set that to "label:_admin". It still does not put ceph.conf into /etc/ceph when adding the label _admin. Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Cephadm orchestrator and special label _admin in 17.2.7
Is your admin keyring under management? The docs say [1]: When a client keyring is placed under management, cephadm will: build a list of target hosts based on the specified placement spec (see Daemon Placement) store a copy of the /etc/ceph/ceph.conf file on the specified host(s) In my case the keyring is under management: quincy-2:~ # ceph orch client-keyring ls ENTITYPLACEMENT MODE OWNER PATH client.admin label:_admin rw--- 0:0 /etc/ceph/ceph.client.admin.keyring [1] https://docs.ceph.com/en/quincy/cephadm/operations/#etc-ceph-ceph-conf Zitat von Robert Sander : Hi, On 1/18/24 14:07, Eugen Block wrote: I just tired that in my test cluster, removed the ceph.conf and admin keyring from /etc/ceph and then added the _admin label to the host via 'ceph orch' and both were created immediately. This is strange, I only get this: 2024-01-18T11:47:07.870079+0100 mgr.cephtest32.ybltym [INF] Added label _admin to host cephtest23 2024-01-18T11:47:07.878786+0100 mgr.cephtest32.ybltym [INF] Updating cephtest23:/var/lib/ceph/ba37db20-2b13-11eb-b8a9-871ba11409f6/config/ceph.conf 2024-01-18T11:47:08.045347+0100 mgr.cephtest32.ybltym [INF] Updating cephtest23:/etc/ceph/ceph.client.admin.keyring 2024-01-18T11:47:08.212303+0100 mgr.cephtest32.ybltym [INF] Updating cephtest23:/var/lib/ceph/ba37db20-2b13-11eb-b8a9-871ba11409f6/config/ceph.client.admin.keyring Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Cephadm orchestrator and special label _admin in 17.2.7
Hi, On 1/18/24 14:07, Eugen Block wrote: I just tired that in my test cluster, removed the ceph.conf and admin keyring from /etc/ceph and then added the _admin label to the host via 'ceph orch' and both were created immediately. This is strange, I only get this: 2024-01-18T11:47:07.870079+0100 mgr.cephtest32.ybltym [INF] Added label _admin to host cephtest23 2024-01-18T11:47:07.878786+0100 mgr.cephtest32.ybltym [INF] Updating cephtest23:/var/lib/ceph/ba37db20-2b13-11eb-b8a9-871ba11409f6/config/ceph.conf 2024-01-18T11:47:08.045347+0100 mgr.cephtest32.ybltym [INF] Updating cephtest23:/etc/ceph/ceph.client.admin.keyring 2024-01-18T11:47:08.212303+0100 mgr.cephtest32.ybltym [INF] Updating cephtest23:/var/lib/ceph/ba37db20-2b13-11eb-b8a9-871ba11409f6/config/ceph.client.admin.keyring Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Cephadm orchestrator and special label _admin in 17.2.7
Hi,
I just tired that in my test cluster, removed the ceph.conf and admin
keyring from /etc/ceph and then added the _admin label to the host via
'ceph orch' and both were created immediately.
# no label for quincy-2
quincy-1:~ # ceph orch host ls
HOST ADDR LABELS STATUS
quincy-1 172.17.2.252 _admin
quincy-2 172.17.2.13
2 hosts in cluster
# moved existing files
quincy-2:~ # mv /etc/ceph/ceph.client.admin.keyring
/etc/ceph/xxxadminxxx.keyring
quincy-2:~ # mv /etc/ceph/ceph.conf /etc/ceph/conf
# added label
quincy-1:~ # ceph orch host label add quincy-2 _admin
Added label _admin to host quincy-2
# mgr deploys files
Jan 18 14:05:52 quincy-2 ceph-mgr[3880499]: log_channel(cephadm) log
[INF] : Added label _admin to host quincy-2
Jan 18 14:05:52 quincy-2 ceph-mgr[3880499]: log_channel(cephadm) log
[INF] : Updating quincy-2:/etc/ceph/ceph.conf
Jan 18 14:05:53 quincy-2 ceph-mgr[3880499]: log_channel(cephadm) log
[INF] : Updating
quincy-2:/var/lib/ceph/1e6e5cb6-73e8-11ee-b195-fa163ee43e22/config/ceph.conf
Jan 18 14:05:54 quincy-2 ceph-mgr[3880499]: log_channel(cephadm) log
[INF] : Updating quincy-2:/etc/ceph/ceph.client.admin.keyring
Jan 18 14:05:54 quincy-2 ceph-mgr[3880499]: log_channel(cephadm) log
[INF] : Updating
quincy-2:/var/lib/ceph/1e6e5cb6-73e8-11ee-b195-fa163ee43e22/config/ceph.client.admin.keyring
quincy-2:~ # ll /etc/ceph/
insgesamt 24
-rw--- 1 root root 151 18. Jan 14:05 ceph.client.admin.keyring
-rw-r--r-- 1 root root 175 18. Jan 14:05 ceph.conf
-rw-r--r-- 1 root root 595 26. Okt 13:17 ceph.pub
-rw-r--r-- 1 root root 175 26. Okt 13:17 conf
-rw-r--r-- 1 root root 92 7. Aug 14:32 rbdmap
-rw--- 1 root root 151 26. Okt 13:17 xxxadminxxx.keyring
This is on 17.2.7:
quincy-2:~ # ceph versions
{
...
"overall": {
"ceph version 17.2.7
(b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable)": 10
}
}
Zitat von Robert Sander :
Hi,
According to the documentation¹ the special host label _admin
instructs the cephadm orchestrator to place a valid ceph.conf and
the ceph.client.admin.keyring into /etc/ceph of the host.
I noticed that (at least) on 17.2.7 only the keyring file is placed
in /etc/ceph, but not ceph.conf.
Both files are placed into the /var/lib/ceph//config directory.
Has something changed?
¹:
https://docs.ceph.com/en/quincy/cephadm/host-management/#special-host-labels
Regards
--
Robert Sander
Heinlein Consulting GmbH
Schwedter Str. 8/9b, 10119 Berlin
https://www.heinlein-support.de
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Amtsgericht Berlin-Charlottenburg - HRB 220009 B
Geschäftsführer: Peer Heinlein - Sitz: Berlin
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
