[ceph-users] Re: [EXTERNAL] How to change RGW certificate in Cephadm?
Do you have an ingress service for HAProxy/keepalived? If so, that’s the service that you will need to have orch redeploy/restart. If not, maybe try `ceph orch redeploy pech` ? Thank you, Josh Beaman From: Kai Stian Olstad Date: Wednesday, June 14, 2023 at 7:58 AM To: ceph-users@ceph.io Subject: [EXTERNAL] [ceph-users] How to change RGW certificate in Cephadm? When I enabled RGW in cephadm I used this spec file rgw.yml service_type: rgw service_id: pech placement: label: cog spec: ssl: true rgw_frontend_ssl_certificate: | -BEGIN CERTIFICATE- -END CERTIFICATE- -BEGIN CERTIFICATE- -END CERTIFICATE- -BEGIN CERTIFICATE- -END CERTIFICATE- -BEGIN RSA PRIVATE KEY- -END RSA PRIVATE KEY- And enabled it with ceph orch apply -i /etc/ceph/rgw.yml The certificate is about to expire so I would like to update it. I updated rgw.yml spec with the new certificate and run ceph orch apply -i /etc/ceph/rgw.yml But nothing happened, so I tried to redeploy one of them with ceph orch daemon redeploy rgw.pech.pech-mon-3.upnvrd It redeployed the RGW, but still uses the old certificate. ceph config-key list | grep rgw gives me two keys of interest mgr/cephadm/spec.rgw.pech and rgw/cert/rgw.pech The content of mgr/cephadm/spec.rgw.pech is the new spec file with the updated certificates, but the rgw/cert/rgw.pech only contains certificate and private key, but the certificate is the old ones about to expire. I have looked in the documentation and can't find how to update the certificate for RGW. Can anyone shed some light on how to replace the certificate? -- Kai Stian Olstad ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: [EXTERNAL] How to change RGW certificate in Cephadm?
On Wed, Jun 14, 2023 at 03:43:17PM +, Beaman, Joshua wrote: Do you have an ingress service for HAProxy/keepalived? If so, that’s the service that you will need to have orch redeploy/restart. If not, maybe try `ceph orch redeploy pech` ? No ingress, but we did have it running at one time with spec file service_type: ingress service_id: rgw.pech This was removed a while ago with ceph orch rm ingress.rgw.pech because haproxy did not have sane values for our environment, timeout was to low and it was hard coded. We then applied the spec file in my previous mail. So we are only running multiple RGW with SSL. Load balancing and HA is done with PowerDNS with LUA-records. ceph orch redeploy pech only gives me an error pech is not a valid daemon name We have a servie named rgw.pech ceph orch ls --service_name=rgw.pech NAME PORTS RUNNING REFRESHED AGE PLACEMENT rgw.pech ?:443 7/7 4m ago 22h label:cog But running ceph orch redeploy rgw.pech will redeploy all 7 RGW, and would be the same as ceph orch daemon redeploy rgw.pech.pech-mon-3.upnvrd but only redeploy one of them. From: Kai Stian Olstad The certificate is about to expire so I would like to update it. I updated rgw.yml spec with the new certificate and run ceph orch apply -i /etc/ceph/rgw.yml But nothing happened, so I tried to redeploy one of them with ceph orch daemon redeploy rgw.pech.pech-mon-3.upnvrd It redeployed the RGW, but still uses the old certificate. ceph config-key list | grep rgw gives me two keys of interest mgr/cephadm/spec.rgw.pech and rgw/cert/rgw.pech The content of mgr/cephadm/spec.rgw.pech is the new spec file with the updated certificates, but the rgw/cert/rgw.pech only contains certificate and private key, but the certificate is the old ones about to expire. When I run ceph orch daemon redeploy rgw.pech.pech-mon-3.upnvrd The log says it using rgw/cert/rgw.pech witch contains the old certificate. 0 framework: beast 0 framework conf key: ssl_port, val: 443 0 framwwork conf key: ssl_certificate, val: config://rgw/cert/rgw.pech -- Kai Stian Olstad ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: [EXTERNAL] How to change RGW certificate in Cephadm?
We resolved our HAProxy woes by creating a custom jinja2 template and deploying as: ceph config-key set mgr/cephadm/services/ingress/haproxy.cfg -i /tmp/haproxy.cfg.j2 But we redeploy new certs the same way you described, and then: ceph orch reconfig ingress.rgw.default.default ceph orch restart rgw.default.default This is all done in the same ansible playbook we use to do initial deployment, but I don’t see anything else in there that looks like it would be needed to update the certs. Best of luck, Josh Beaman From: Kai Stian Olstad Date: Thursday, June 15, 2023 at 2:47 AM To: Beaman, Joshua Cc: ceph-users@ceph.io Subject: Re: [EXTERNAL] [ceph-users] How to change RGW certificate in Cephadm? On Wed, Jun 14, 2023 at 03:43:17PM +, Beaman, Joshua wrote: >Do you have an ingress service for HAProxy/keepalived? If so, that’s the >service that you will need to have orch redeploy/restart. If not, maybe try >`ceph orch redeploy pech` ? No ingress, but we did have it running at one time with spec file service_type: ingress service_id: rgw.pech This was removed a while ago with ceph orch rm ingress.rgw.pech because haproxy did not have sane values for our environment, timeout was to low and it was hard coded. We then applied the spec file in my previous mail. So we are only running multiple RGW with SSL. Load balancing and HA is done with PowerDNS with LUA-records. ceph orch redeploy pech only gives me an error pech is not a valid daemon name We have a servie named rgw.pech ceph orch ls --service_name=rgw.pech NAME PORTS RUNNING REFRESHED AGE PLACEMENT rgw.pech ?:443 7/7 4m ago 22h label:cog But running ceph orch redeploy rgw.pech will redeploy all 7 RGW, and would be the same as ceph orch daemon redeploy rgw.pech.pech-mon-3.upnvrd but only redeploy one of them. >From: Kai Stian Olstad >The certificate is about to expire so I would like to update it. >I updated rgw.yml spec with the new certificate and run > ceph orch apply -i /etc/ceph/rgw.yml > >But nothing happened, so I tried to redeploy one of them with > ceph orch daemon redeploy rgw.pech.pech-mon-3.upnvrd > >It redeployed the RGW, but still uses the old certificate. > > > ceph config-key list | grep rgw >gives me two keys of interest mgr/cephadm/spec.rgw.pech and rgw/cert/rgw.pech > >The content of mgr/cephadm/spec.rgw.pech is the new spec file with the updated >certificates, but the rgw/cert/rgw.pech only contains certificate and private >key, but the certificate is the old ones about to expire. When I run ceph orch daemon redeploy rgw.pech.pech-mon-3.upnvrd The log says it using rgw/cert/rgw.pech witch contains the old certificate. 0 framework: beast 0 framework conf key: ssl_port, val: 443 0 framwwork conf key: ssl_certificate, val: config://rgw/cert/rgw.pech -- Kai Stian Olstad ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: [EXTERNAL] How to change RGW certificate in Cephadm?
On Thu, Jun 15, 2023 at 03:58:40PM +, Beaman, Joshua wrote: We resolved our HAProxy woes by creating a custom jinja2 template and deploying as: ceph config-key set mgr/cephadm/services/ingress/haproxy.cfg -i /tmp/haproxy.cfg.j2 Thanks, wish I knew that a few month ago before I threw out ingress. But we redeploy new certs the same way you described, and then: ceph orch reconfig ingress.rgw.default.default ceph orch restart rgw.default.default This is all done in the same ansible playbook we use to do initial deployment, but I don’t see anything else in there that looks like it would be needed to update the certs. After testing this I will claim this is a bug. The first time "ceph orch apply -i /etc/ceph/rgw.yml" is run it creates to keys mgr/cephadm/spex.rgw.pech and rgw/cert/rgw.pech But later when the spec file is updated and apply is run again only mgr/cephadm/spex.rgw.pech is updated. When the RGW start the log says it using the certificate in rgw/cert/rgw.pech So, if I read out the certificate from mgr/cephadm/spex.rgw.pech and add that in rgw/cert/rgw.pech and then restart the RGW it picks up the new certificate. The command to do this ceph config-key get mgr/cephadm/spex.rgw.pech | jq -r .spec.spec.rgw_frontend_ssl_certificate | ceph config-key set rgw/cert/rgw.pech - ceph orch restart rgw.pech My claim is that Ceph should update "rgw/cert/rgw.pech" when "mgr/cephadm/spex.rgw.pech" is updated. -- Kai Stian Olstad ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: [EXTERNAL] How to change RGW certificate in Cephadm?
Nice find! Totally looks buggy. Also thanks for sharing that command…I love a good one-liner! Josh Beaman From: Kai Stian Olstad Date: Friday, June 16, 2023 at 7:35 AM To: Beaman, Joshua Cc: ceph-users@ceph.io Subject: Re: [EXTERNAL] [ceph-users] How to change RGW certificate in Cephadm? On Thu, Jun 15, 2023 at 03:58:40PM +, Beaman, Joshua wrote: >We resolved our HAProxy woes by creating a custom jinja2 template and >deploying as: >ceph config-key set mgr/cephadm/services/ingress/haproxy.cfg -i >/tmp/haproxy.cfg.j2 Thanks, wish I knew that a few month ago before I threw out ingress. >But we redeploy new certs the same way you described, and then: >ceph orch reconfig ingress.rgw.default.default >ceph orch restart rgw.default.default > >This is all done in the same ansible playbook we use to do initial deployment, >but I don’t see anything else in there that looks like it would be needed to >update the certs. After testing this I will claim this is a bug. The first time "ceph orch apply -i /etc/ceph/rgw.yml" is run it creates to keys mgr/cephadm/spex.rgw.pech and rgw/cert/rgw.pech But later when the spec file is updated and apply is run again only mgr/cephadm/spex.rgw.pech is updated. When the RGW start the log says it using the certificate in rgw/cert/rgw.pech So, if I read out the certificate from mgr/cephadm/spex.rgw.pech and add that in rgw/cert/rgw.pech and then restart the RGW it picks up the new certificate. The command to do this ceph config-key get mgr/cephadm/spex.rgw.pech | jq -r .spec.spec.rgw_frontend_ssl_certificate | ceph config-key set rgw/cert/rgw.pech - ceph orch restart rgw.pech My claim is that Ceph should update "rgw/cert/rgw.pech" when "mgr/cephadm/spex.rgw.pech" is updated. -- Kai Stian Olstad ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
