On 2017-09-29 11:31, Maged Mokhtar wrote:
> On 2017-09-29 10:44, Adrian Saul wrote:
>
> Do you mean that after you delete and remove the crush and auth entries for
> the OSD, when you go to create another OSD later it will re-use the previous
> OSD ID that you have destroyed in the past?
>
> Because I have seen that behaviour as well - but only for previously
> allocated OSD IDs that have been osd rm/crush rm/auth del.
>
> -----Original Message-----
> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
> Luis Periquito
> Sent: Friday, 29 September 2017 6:01 PM
> To: Ceph Users <ceph-users@lists.ceph.com>
> Subject: [ceph-users] osd create returns duplicate ID's
>
> Hi all,
>
> I use puppet to deploy and manage my clusters.
>
> Recently, as I have been doing a removal of old hardware and adding of new
> I've noticed that sometimes the "ceph osd create" is returning repeated IDs.
> Usually it's on the same server, but yesterday I saw it in different servers.
>
> I was expecting the OSD ID's to be unique, and when they come on the same
> server puppet starts spewing errors - which is desirable - but when it's in
> different servers it broke those OSDs in Ceph. As they hadn't backfill any
> full
> PGs I just wiped, removed and started anew.
>
> As for the process itself: The OSDs are marked out and removed from crush,
> when empty they are auth del and osd rm. After building the server puppet
> will osd create, and use the generated ID for crush move and mkfs.
>
> Unfortunately I haven't been able to reproduce in isolation, and being a
> production cluster logging is tuned way down.
>
> This has happened in several different clusters, but they are all running
> 10.2.7.
>
> Any ideas?
>
> thanks,
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com Confidentiality: This
> email and any attachments are confidential and may be subject to copyright,
> legal or some other professional privilege. They are intended solely for the
> attention and use of the named addressee(s). They may only be copied,
> distributed or disclosed with the consent of the copyright owner. If you have
> received this email by mistake or by breach of the confidentiality clause,
> please notify the sender immediately by return email and delete or destroy
> all copies of the email. Any confidentiality, privilege or copyright is not
> waived or lost because this email has been sent to you by mistake.
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Ceph will re-use previous deleted osd ids. this is desirable to minimize
data rebalancing. What is not correct is having duplicate active ids and
i am not sure how this is happening but i would suggest avoid add/remove
osds simultaneously ie should add them one at a time, if you can do it
manually check that the osd was added in crush and process is up and
running before trying to add a new one..if that still produces
duplicates then there is a serious issue. If adding via script double
check it is not trying to do several tasks at once.
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
One more thing if you are using a script to add osds, try to add a small
sleep/pause to allow the new osd to get activated via udev and register
itself in crush before starting to create a new one.
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
