On Fri, Aug 13, 2021 at 9:45 AM Boris Behrens <b...@kervyn.de> wrote:
>
> Hi Janne,
> thanks for the hint. I was aware of that, but it is goot to add that
> knowledge to the question for further googlesearcher.
>
> Hi Ilya,
> that fixed it. Do we know why the discard does not work when the partition
> table is not aligned? We provide OS templates to our customer, but they can
> also create and attach an empty block device, and they will certainly not
> check if the partitions are aligned correctly.
Hi Boris,
Not the partition table but the partitions themselves.
If the partition isn't aligned to rbd object size (but still aligned to
something reasonable such as 1M), discard would work but *appear* to be
ineffective because for large discard requests instead of just removing
whole objects which is immediately visible in "rbd du" output it would
have to resort to truncating and punching holes in those objects. To
see some of the effect one would need to run "rbd du --exact" which is
slow.
Some of because "rbd du --exact" would basically sum up the remaining
truncated object sizes but since holes don't consume space that figure
would still be inaccurate (bigger). To see the real effect, one would
need to look at "ceph osd df" DATA column before running discard (here
removing the file) and after.
And what Janne mentioned, of course. "-o discard" isn't guaranteed
to actually free up space in all scenarios. This is why I said "for
discard to work the same way as without partitioning" in my previous
email -- because irrespective of partitioning online discards wouldn't
always have the desired behaviour (on any block device, not just rbd).
Thanks,
Ilya
>
> Cheers
> Boris
>
>
> Am Fr., 13. Aug. 2021 um 08:44 Uhr schrieb Janne Johansson <
> icepic...@gmail.com>:
>
> > Den tors 12 aug. 2021 kl 17:04 skrev Boris Behrens <b...@kervyn.de>:
> > > Hi everybody,
> > > we just stumbled over a problem where the rbd image does not shrink, when
> > > files are removed.
> > > This only happenes when the rbd image is partitioned.
> > >
> > > * We tested it with centos8/ubuntu20.04 with ext4 and a gpt partition
> > table
> > > (/boot and /)
> > > * the kvm device is virtio-scsi-pci with krbd
> > > * Mount option discard is set
> > > * command to create large file: dd if=/dev/zero of=testfile bs=64M
> > > count=1000
> > > * the image grows in the size we expect
> > > * when we remove the testfile the rbd image stays at the size
> > > * we wen recreate the deleted file with the command the rbd image grows
> > > further
> >
> > Just a small nit on this single point, regardless of if trim/discard
> > works or not:
> > There is no guarantee that writing a file, removing it and then
> > re-writing a file
> > will ever end up in the same spot again. In fact, most modern filesystems
> > will
> > probably make sure to NOT place things at the same spot again.
> > Since the second write ends up in a different place, it will once again
> > expand
> > your sparse/thin image by the amount of written bytes, this is very much
> > to be expected.
> >
> > I'm sorry if you already knew this and I am just stating the obvious to
> > you, but
> > your text came over as if you expected the second write to not increase the
> > image since that "space" was already blown up on the first write.
> >
> > Trim/discard should still be investigated so you can make it shrink back
> > again somehow, just wanted to point this out for the records.
> >
> >
> > --
> > May the most significant bit of your life be positive.
> >
>
>
> --
> Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im
> groüen Saal.
> _______________________________________________
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
