[ceph-users] Re: import OSD after host OS reinstallation
Thank you Eugen for looking into it! In short, it works. I'm using 16.2.10. What I did wrong was to remove the OSD, which makes no sense. Tony From: Eugen Block Sent: April 28, 2023 06:46 AM To: ceph-users@ceph.io Subject: [ceph-users] Re: import OSD after host OS reinstallation I chatted with Mykola who helped me get the OSDs back up. My test cluster was on 16.2.5 (and still mostly is), after upgrading only the MGRs to a more recent version (16.2.10) the activate command worked successfully and the existing OSDs got back up. Not sure if that's a bug or something else, but which exact versions are you using? Zitat von Eugen Block : > I found a small two-node cluster to test this on pacific, I can > reproduce it. After reinstalling the host (VM) most of the other > services are redeployed (mon, mgr, mds, crash), but not the OSDs. I > will take a closer look. > > Zitat von Tony Liu : > >> Tried [1] already, but got error. >> Created no osd(s) on host ceph-4; already created? >> >> The error is from [2] in deploy_osd_daemons_for_existing_osds(). >> >> Not sure what's missing. >> Should OSD be removed, or removed with --replace, or untouched >> before host reinstallation? >> >> [1] >> https://docs.ceph.com/en/pacific/cephadm/services/osd/#activate-existing-osds >> [2] >> https://github.com/ceph/ceph/blob/0a5b3b373b8a5ba3081f1f110cec24d82299cac8/src/pybind/mgr/cephadm/services/osd.py#L196 >> >> Thanks! >> Tony >> >> From: Tony Liu >> Sent: April 27, 2023 10:20 PM >> To: ceph-users@ceph.io; d...@ceph.io >> Subject: [ceph-users] import OSD after host OS reinstallation >> >> Hi, >> >> The cluster is with Pacific and deployed by cephadm on container. >> The case is to import OSDs after host OS reinstallation. >> All OSDs are SSD who has DB/WAL and data together. >> Did some research, but not able to find a working solution. >> Wondering if anyone has experiences in this? >> What needs to be done before host OS reinstallation and what's after? >> >> >> Thanks! >> Tony >> ___ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-le...@ceph.io >> ___ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: import OSD after host OS reinstallation
I chatted with Mykola who helped me get the OSDs back up. My test cluster was on 16.2.5 (and still mostly is), after upgrading only the MGRs to a more recent version (16.2.10) the activate command worked successfully and the existing OSDs got back up. Not sure if that's a bug or something else, but which exact versions are you using? Zitat von Eugen Block : I found a small two-node cluster to test this on pacific, I can reproduce it. After reinstalling the host (VM) most of the other services are redeployed (mon, mgr, mds, crash), but not the OSDs. I will take a closer look. Zitat von Tony Liu : Tried [1] already, but got error. Created no osd(s) on host ceph-4; already created? The error is from [2] in deploy_osd_daemons_for_existing_osds(). Not sure what's missing. Should OSD be removed, or removed with --replace, or untouched before host reinstallation? [1] https://docs.ceph.com/en/pacific/cephadm/services/osd/#activate-existing-osds [2] https://github.com/ceph/ceph/blob/0a5b3b373b8a5ba3081f1f110cec24d82299cac8/src/pybind/mgr/cephadm/services/osd.py#L196 Thanks! Tony From: Tony Liu Sent: April 27, 2023 10:20 PM To: ceph-users@ceph.io; d...@ceph.io Subject: [ceph-users] import OSD after host OS reinstallation Hi, The cluster is with Pacific and deployed by cephadm on container. The case is to import OSDs after host OS reinstallation. All OSDs are SSD who has DB/WAL and data together. Did some research, but not able to find a working solution. Wondering if anyone has experiences in this? What needs to be done before host OS reinstallation and what's after? Thanks! Tony ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: import OSD after host OS reinstallation
I found a small two-node cluster to test this on pacific, I can reproduce it. After reinstalling the host (VM) most of the other services are redeployed (mon, mgr, mds, crash), but not the OSDs. I will take a closer look. Zitat von Tony Liu : Tried [1] already, but got error. Created no osd(s) on host ceph-4; already created? The error is from [2] in deploy_osd_daemons_for_existing_osds(). Not sure what's missing. Should OSD be removed, or removed with --replace, or untouched before host reinstallation? [1] https://docs.ceph.com/en/pacific/cephadm/services/osd/#activate-existing-osds [2] https://github.com/ceph/ceph/blob/0a5b3b373b8a5ba3081f1f110cec24d82299cac8/src/pybind/mgr/cephadm/services/osd.py#L196 Thanks! Tony From: Tony Liu Sent: April 27, 2023 10:20 PM To: ceph-users@ceph.io; d...@ceph.io Subject: [ceph-users] import OSD after host OS reinstallation Hi, The cluster is with Pacific and deployed by cephadm on container. The case is to import OSDs after host OS reinstallation. All OSDs are SSD who has DB/WAL and data together. Did some research, but not able to find a working solution. Wondering if anyone has experiences in this? What needs to be done before host OS reinstallation and what's after? Thanks! Tony ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: import OSD after host OS reinstallation
Hi, Not sure what's missing. Should OSD be removed, or removed with --replace, or untouched before host reinstallation? If you want to reuse existing OSDs why would you remove them? That's the whole point of reusing them after installation. Tried [1] already, but got error. Created no osd(s) on host ceph-4; already created? That is expected, you don't want to create new OSDs but just activate the existing ones. Do you see cephadm trying to activate the OSDs? Check /var/log/ceph/cephadm.log on the reinstalled host for more details, maybe the mgr log has some information as well. Regards, Eugen Zitat von Tony Liu : Tried [1] already, but got error. Created no osd(s) on host ceph-4; already created? The error is from [2] in deploy_osd_daemons_for_existing_osds(). Not sure what's missing. Should OSD be removed, or removed with --replace, or untouched before host reinstallation? [1] https://docs.ceph.com/en/pacific/cephadm/services/osd/#activate-existing-osds [2] https://github.com/ceph/ceph/blob/0a5b3b373b8a5ba3081f1f110cec24d82299cac8/src/pybind/mgr/cephadm/services/osd.py#L196 Thanks! Tony From: Tony Liu Sent: April 27, 2023 10:20 PM To: ceph-users@ceph.io; d...@ceph.io Subject: [ceph-users] import OSD after host OS reinstallation Hi, The cluster is with Pacific and deployed by cephadm on container. The case is to import OSDs after host OS reinstallation. All OSDs are SSD who has DB/WAL and data together. Did some research, but not able to find a working solution. Wondering if anyone has experiences in this? What needs to be done before host OS reinstallation and what's after? Thanks! Tony ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: import OSD after host OS reinstallation
Tried [1] already, but got error. Created no osd(s) on host ceph-4; already created? The error is from [2] in deploy_osd_daemons_for_existing_osds(). Not sure what's missing. Should OSD be removed, or removed with --replace, or untouched before host reinstallation? [1] https://docs.ceph.com/en/pacific/cephadm/services/osd/#activate-existing-osds [2] https://github.com/ceph/ceph/blob/0a5b3b373b8a5ba3081f1f110cec24d82299cac8/src/pybind/mgr/cephadm/services/osd.py#L196 Thanks! Tony From: Tony Liu Sent: April 27, 2023 10:20 PM To: ceph-users@ceph.io; d...@ceph.io Subject: [ceph-users] import OSD after host OS reinstallation Hi, The cluster is with Pacific and deployed by cephadm on container. The case is to import OSDs after host OS reinstallation. All OSDs are SSD who has DB/WAL and data together. Did some research, but not able to find a working solution. Wondering if anyone has experiences in this? What needs to be done before host OS reinstallation and what's after? Thanks! Tony ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
