Re: [ceph-users] ceph storage capacity does not free when deleting contents from RBD volumes

[Christian Balzer]

Hello,

On Fri, 20 May 2016 00:11:02 + David Turner wrote:

> You can also mount the rbd with the discard option. It works the same
> way as you would mount an ssd to free up the space when you delete
> things. I use the discard option on my ext4 rbds on Ubuntu and it frees
> up the used Ceph space immediately.
> 
While that certainly works (and has the advantage of being fully automatic
and not needing any human or CRON intervention), it has a big disadvantage
as well.

With normal storage (SSDs) the consensus already is that manual fstrim at a
time of low utilization is preferable, since TRIM activity can slow down
things as the SSD does its housekeeping.

With Ceph this is even more pronounced (TRIM'ing RBD images is quite
costly in terms of IO, IOPS), so doing TRIMs only when actually needed and
during quite times is much preferred.

Regards,

Christian
> Sent from my iPhone
> 
> On May 19, 2016, at 12:30 PM, Albert Archer
> > wrote:
> 
> Thank you for your great support .
> 
> Best Regards
> Albert
> 
> On Thu, May 19, 2016 at 10:41 PM, Udo Lembke
> > wrote: Hi Albert,
> to free unused space you must enable trim (or do an fstrim) in the vm -
> and all things in the storage chain must support this. The normal
> virtio-driver don't support trim, but if you use scsi-disks with
> virtio-scsi-driver you can use it. Work well but need some time for huge
> filesystems.
> 
> Udo
> 
> 
> On 19.05.2016 19:58, Albert Archer wrote:
> Hello All.
> I am newbie in ceph. and i use jewel release for testing purpose. it
> seems every thing is OK, HEALTH_OK , all of OSDs are in UP and IN state.
> I create some RBD images (rbd create  ) and map to some ubuntu
> host .
> I can read and write data to my volume , but when i delete some content
> from volume (e,g some huge files,...), populated capacity of cluster
> does not free and None of objects were clean.
> what is the problem ???
> 
> Regards
> [https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif]Albert
> 
> 
> 
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 
> 
> 
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


-- 
Christian BalzerNetwork/Systems Engineer
ch...@gol.com   Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph storage capacity does not free when deleting contents from RBD volumes

[David Turner]

You can also mount the rbd with the discard option. It works the same way as 
you would mount an ssd to free up the space when you delete things. I use the 
discard option on my ext4 rbds on Ubuntu and it frees up the used Ceph space 
immediately.

Sent from my iPhone

On May 19, 2016, at 12:30 PM, Albert Archer 
> wrote:

Thank you for your great support .

Best Regards
Albert

On Thu, May 19, 2016 at 10:41 PM, Udo Lembke 
> wrote:
Hi Albert,
to free unused space you must enable trim (or do an fstrim) in the vm - and all 
things in the storage chain must support this.
The normal virtio-driver don't support trim, but if you use scsi-disks with 
virtio-scsi-driver you can use it.
Work well but need some time for huge filesystems.

Udo


On 19.05.2016 19:58, Albert Archer wrote:
Hello All.
I am newbie in ceph. and i use jewel release for testing purpose. it
seems every thing is OK, HEALTH_OK , all of OSDs are in UP and IN state.
I create some RBD images (rbd create  ) and map to some ubuntu
host .
I can read and write data to my volume , but when i delete some content
from volume (e,g some huge files,...), populated capacity of cluster
does not free and None of objects were clean.
what is the problem ???

Regards
[https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif]Albert



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph storage capacity does not free when deleting contents from RBD volumes

[Albert Archer]

Thank you for your great support .

Best Regards
Albert

On Thu, May 19, 2016 at 10:41 PM, Udo Lembke  wrote:

> Hi Albert,
> to free unused space you must enable trim (or do an fstrim) in the vm -
> and all things in the storage chain must support this.
> The normal virtio-driver don't support trim, but if you use scsi-disks
> with virtio-scsi-driver you can use it.
> Work well but need some time for huge filesystems.
>
> Udo
>
>
> On 19.05.2016 19:58, Albert Archer wrote:
>
> Hello All.
> I am newbie in ceph. and i use jewel release for testing purpose. it
> seems every thing is OK, HEALTH_OK , all of OSDs are in UP and IN state.
> I create some RBD images (rbd create  ) and map to some ubuntu
> host .
> I can read and write data to my volume , but when i delete some content
> from volume (e,g some huge files,...), populated capacity of cluster
> does not free and None of objects were clean.
> what is the problem ???
>
> Regards
> Albert
>
>
> ___
> ceph-users mailing 
> listceph-us...@lists.ceph.comhttp://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph storage capacity does not free when deleting contents from RBD volumes

[Udo Lembke]

Hi Albert,
to free unused space you must enable trim (or do an fstrim) in the vm -
and all things in the storage chain must support this.
The normal virtio-driver don't support trim, but if you use scsi-disks
with virtio-scsi-driver you can use it.
Work well but need some time for huge filesystems.

Udo

On 19.05.2016 19:58, Albert Archer wrote:
> Hello All.
> I am newbie in ceph. and i use jewel release for testing purpose. it
> seems every thing is OK, HEALTH_OK , all of OSDs are in UP and IN state.
> I create some RBD images (rbd create  ) and map to some ubuntu
> host . 
> I can read and write data to my volume , but when i delete some content
> from volume (e,g some huge files,...), populated capacity of cluster
> does not free and None of objects were clean.
> what is the problem ???
>
> Regards 
> Albert
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph storage capacity does not free when deleting contents from RBD volumes

[Edward R Huyer]

That is normal behavior.  Ceph has no understanding of the filesystem living on 
top of the RBD, so it doesn’t know when space is freed up.  If you are running 
a sufficiently current kernel, you can use fstrim to cause the kernel to tell 
Ceph what blocks are free.  More details here:  
http://www.sebastien-han.fr/blog/2015/01/26/ceph-and-krbd-discard/


From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Albert 
Archer
Sent: Thursday, May 19, 2016 1:59 PM
To: ceph-us...@ceph.com
Subject: [ceph-users] ceph storage capacity does not free when deleting 
contents from RBD volumes

Hello All.
I am newbie in ceph. and i use jewel release for testing purpose. it
seems every thing is OK, HEALTH_OK , all of OSDs are in UP and IN state.
I create some RBD images (rbd create  ) and map to some ubuntu
host .
I can read and write data to my volume , but when i delete some content
from volume (e,g some huge files,...), populated capacity of cluster
does not free and None of objects were clean.
what is the problem ???

Regards
[https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif]Albert
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] ceph storage capacity does not free when deleting contents from RBD volumes

[Albert Archer]

Hello All.
I am newbie in ceph. and i use jewel release for testing purpose. it
seems every thing is OK, HEALTH_OK , all of OSDs are in UP and IN state.
I create some RBD images (rbd create  ) and map to some ubuntu
host .
I can read and write data to my volume , but when i delete some content
from volume (e,g some huge files,...), populated capacity of cluster
does not free and None of objects were clean.
what is the problem ???

Regards
Albert
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com