Re: [ceph-users] TRIM / DISCARD run at low priority by the OSDs?
Hi Alexandre, Thanks for the note. I was not clear enough. The fstrim I was running was only on the krbd mountpoints. The backend OSDs only have standard hard disks, not SSDs, so they don't need to be trimmed. Instead I was reclaiming free space as reported by Ceph. Running fstrim on the rbd mountpoints this caused the OSDs to become very busy, affecting all rbds, not just those being trimmed. I was hoping someone had an idea of how to make the OSDs not become busy while running fstrim on the rbd mountpoints. E.g. if Ceph made a distinction between trim operations on RBDs and other types, it could give those operations lower priority. Thanks again! Chad. On Monday, August 24, 2015 18:26:30 you wrote: > Hi, > > I'm not sure for krbd, but with librbd, using trim/discard on the client, > > don't do trim/discard on the osd physical disk. > > It's simply write zeroes in the rbd image. > > zeores write can be skipped since this commit (librbd related) > https://github.com/xiaoxichen/ceph/commit/e7812b8416012141cf8faef577e7b27e1b > 29d5e3 +OPTION(rbd_skip_partial_discard, OPT_BOOL, false) > > > Then you can still manage fstrim manually on the osd servers > > - Mail original - > De: "Chad William Seys" > À: "ceph-users" > Envoyé: Samedi 22 Août 2015 04:26:38 > Objet: [ceph-users] TRIM / DISCARD run at low priority by the OSDs? > > Hi All, > > Is it possible to give TRIM / DISCARD initiated by krbd low priority on the > OSDs? > > I know it is possible to run fstrim at Idle priority on the rbd mount point, > e.g. ionice -c Idle fstrim -v $MOUNT . > > But this Idle priority (it appears) only is within the context of the node > executing fstrim . If the node executing fstrim is Idle then the OSDs are > very busy and performance suffers. > > Is it possible to tell the OSD daemons (or whatever) to perform the TRIMs at > low priority also? > > Thanks! > Chad. > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] TRIM / DISCARD run at low priority by the OSDs?
Hi, I'm not sure for krbd, but with librbd, using trim/discard on the client, don't do trim/discard on the osd physical disk. It's simply write zeroes in the rbd image. zeores write can be skipped since this commit (librbd related) https://github.com/xiaoxichen/ceph/commit/e7812b8416012141cf8faef577e7b27e1b29d5e3 +OPTION(rbd_skip_partial_discard, OPT_BOOL, false) Then you can still manage fstrim manually on the osd servers - Mail original - De: "Chad William Seys" À: "ceph-users" Envoyé: Samedi 22 Août 2015 04:26:38 Objet: [ceph-users] TRIM / DISCARD run at low priority by the OSDs? Hi All, Is it possible to give TRIM / DISCARD initiated by krbd low priority on the OSDs? I know it is possible to run fstrim at Idle priority on the rbd mount point, e.g. ionice -c Idle fstrim -v $MOUNT . But this Idle priority (it appears) only is within the context of the node executing fstrim . If the node executing fstrim is Idle then the OSDs are very busy and performance suffers. Is it possible to tell the OSD daemons (or whatever) to perform the TRIMs at low priority also? Thanks! Chad. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] TRIM / DISCARD run at low priority by the OSDs?
Hi All, Is it possible to give TRIM / DISCARD initiated by krbd low priority on the OSDs? I know it is possible to run fstrim at Idle priority on the rbd mount point, e.g. ionice -c Idle fstrim -v $MOUNT . But this Idle priority (it appears) only is within the context of the node executing fstrim . If the node executing fstrim is Idle then the OSDs are very busy and performance suffers. Is it possible to tell the OSD daemons (or whatever) to perform the TRIMs at low priority also? Thanks! Chad. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
