[ceph-users] Re: Understanding subvolumes

[Robert Sander]

On 01.02.24 00:20, Matthew Melendy wrote:
In our department we're getting starting with Ceph 'reef', using Ceph 
FUSE client for our Ubuntu workstations.


So far so good, except I can't quite figure out one aspect of subvolumes.


AFAIK subvolumes were introduced to be used with Kubernetes and other 
cloud technologies.


If you run a classical file service on top of CephFS you usually do not 
need subvolumes but can go with normal quotas on directories.


Regards
--
Robert Sander
Heinlein Support GmbH
Linux: Akademie - Support - Hosting
http://www.heinlein-support.de

Tel: 030-405051-43
Fax: 030-405051-19

Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein  -- Sitz: Berlin
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Understanding subvolumes

[Kotresh Hiremath Ravishankar]

Comments inline.

On Thu, Feb 1, 2024 at 4:51 AM Matthew Melendy  wrote:

> In our department we're getting starting with Ceph 'reef', using Ceph FUSE
> client for our Ubuntu workstations.
>
> So far so good, except I can't quite figure out one aspect of subvolumes.
>
> When I do the commands:
>
> [root@ceph1 ~]# ceph fs subvolumegroup create cephfs csvg
> [root@ceph1 ~]# ceph fs subvolume create cephfs staff csvg --size
> 2
>
> I get these results:
>
> - A subvolume group csvg is created on volume cephfs
> - A subvolume called staff is created in csvg subvolume (like
> /volumes/csvg/staff ) however there is no size limit set at this folder in
> the Ceph dashboard view
> - A folder with an random UUID name is created inside the subvolume staff
> ( like /volumes/csvg/staff/6a1b3de5-f6ab-4878-aea3-3c3c6f96ffcf ); this
> folder does have a size set on it of 2TB
>
> My questions are:
> - what's the purpose of this UUID, and is it a requirement?
>

The UUID directory is essentially the data directory for the user to store
data.
The subvolume directory is used internally to store metadata related to
subvolume
to support all the subvolume operations.

For more detailed information, please go through the following comment in
the code.
https://github.com/ceph/ceph/blob/main/src/pybind/mgr/volumes/fs/operations/versions/subvolume_v2.py#L19C1-L38C8


- which directory should be mounted for my clients, staff/ or staff/{UUID},
> for the size limit to take effect?
>

 The quota (size passed during subvolume creation/or set after creation) is
enforced on the uuid directory not on subvolume
directory. So it should be staff/{uuid}. The idea is to use the 'subvolume
getpath' command and use the returned path to mount. That
should take care of all the things.

- is there any way to hide or disable this UUID for client mounts? (eg in
> /etc/fstab)
>

I didn't quite get this ?


> [root@ceph1 ~]# ceph fs subvolume getpath cephfs staff csvg
> /volumes/csvg/staff/6a1b3de5-f6ab-4878-aea3-3c3c6f96ffcf
>
> [root@ceph1 ~]# ceph fs subvolume ls cephfs csvg
> [
>  {
>  "name": "staff"
>  }
> ]
>
>
>
> --
> Sincerely,
>
>
> Matthew Melendy
>
> IT Services Specialist
> CS System Services Group
> FEC 3550, University of New Mexico
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Understanding subvolumes

[Neeraj Pratap Singh]

Hi,
In reply to your question for this UUID thing, we do need it for CLONING
subvolume and yes, it is a requirement.
And the subvolume mount path is the entire directory path.

On Thu, Feb 1, 2024 at 4:51 AM Matthew Melendy  wrote:

> In our department we're getting starting with Ceph 'reef', using Ceph FUSE
> client for our Ubuntu workstations.
>
> So far so good, except I can't quite figure out one aspect of subvolumes.
>
> When I do the commands:
>
> [root@ceph1 ~]# ceph fs subvolumegroup create cephfs csvg
> [root@ceph1 ~]# ceph fs subvolume create cephfs staff csvg --size
> 2
>
> I get these results:
>
> - A subvolume group csvg is created on volume cephfs
> - A subvolume called staff is created in csvg subvolume (like
> /volumes/csvg/staff ) however there is no size limit set at this folder in
> the Ceph dashboard view
> - A folder with an random UUID name is created inside the subvolume staff
> ( like /volumes/csvg/staff/6a1b3de5-f6ab-4878-aea3-3c3c6f96ffcf ); this
> folder does have a size set on it of 2TB
>
> My questions are:
> - what's the purpose of this UUID, and is it a requirement?
> - which directory should be mounted for my clients, staff/ or
> staff/{UUID}, for the size limit to take effect?
> - is there any way to hide or disable this UUID for client mounts? (eg in
> /etc/fstab)
>
> [root@ceph1 ~]# ceph fs subvolume getpath cephfs staff csvg
> /volumes/csvg/staff/6a1b3de5-f6ab-4878-aea3-3c3c6f96ffcf
>
> [root@ceph1 ~]# ceph fs subvolume ls cephfs csvg
> [
>  {
>  "name": "staff"
>  }
> ]
>
>
>
> --
> Sincerely,
>
>
> Matthew Melendy
>
> IT Services Specialist
> CS System Services Group
> FEC 3550, University of New Mexico
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io