On 2017-09-29 10:44, Adrian Saul wrote:
> Do you mean that after you delete and remove the crush and auth entries for
> the OSD, when you go to create another OSD later it will re-use the previous
> OSD ID that you have destroyed in the past?
>
> Because I have seen that behaviour as well - but only for previously
> allocated OSD IDs that have been osd rm/crush rm/auth del.
>
>> -----Original Message-----
>> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
>> Luis Periquito
>> Sent: Friday, 29 September 2017 6:01 PM
>> To: Ceph Users <ceph-users@lists.ceph.com>
>> Subject: [ceph-users] osd create returns duplicate ID's
>>
>> Hi all,
>>
>> I use puppet to deploy and manage my clusters.
>>
>> Recently, as I have been doing a removal of old hardware and adding of new
>> I've noticed that sometimes the "ceph osd create" is returning repeated IDs.
>> Usually it's on the same server, but yesterday I saw it in different servers.
>>
>> I was expecting the OSD ID's to be unique, and when they come on the same
>> server puppet starts spewing errors - which is desirable - but when it's in
>> different servers it broke those OSDs in Ceph. As they hadn't backfill any
>> full
>> PGs I just wiped, removed and started anew.
>>
>> As for the process itself: The OSDs are marked out and removed from crush,
>> when empty they are auth del and osd rm. After building the server puppet
>> will osd create, and use the generated ID for crush move and mkfs.
>>
>> Unfortunately I haven't been able to reproduce in isolation, and being a
>> production cluster logging is tuned way down.
>>
>> This has happened in several different clusters, but they are all running
>> 10.2.7.
>>
>> Any ideas?
>>
>> thanks,
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> Confidentiality: This email and any attachments are confidential and may be
> subject to copyright, legal or some other professional privilege. They are
> intended solely for the attention and use of the named addressee(s). They may
> only be copied, distributed or disclosed with the consent of the copyright
> owner. If you have received this email by mistake or by breach of the
> confidentiality clause, please notify the sender immediately by return email
> and delete or destroy all copies of the email. Any confidentiality, privilege
> or copyright is not waived or lost because this email has been sent to you by
> mistake.
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Ceph will re-use previous deleted osd ids. this is desirable to minimize
data rebalancing. What is not correct is having duplicate active ids and
i am not sure how this is happening but i would suggest avoid add/remove
osds simultaneously ie should add them one at a time, if you can do it
manually check that the osd was added in crush and process is up and
running before trying to add a new one..if that still produces
duplicates then there is a serious issue. If adding via script double
check it is not trying to do several tasks at once.
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
