Hello, I'm wondering if it's possible to grow a volume (such as in a cloud/VM environment) and use pvresize/lvextend to utilize the extra space in my pool.
I am testing with the following environment: * Running on cloud provider (Google Cloud) * 3 nodes, 1 OSD each * 1 storage pool with "size" of 3 (data replicated on all nodes) * Initial disk size of 100 GB on each node, initialized as bluestore OSDs I grew all three volumes (100 GB -> 150 GB) being used as OSDs in the Google console. Then used pvresize/lvextend on all devices and rebooted all nodes one-by-one. In the end, the nodes are somewhat recognizing the additional space, but it's showing up as being utilized. Before resize (there's ~1 GB of data in my pool): $ ceph -s cluster: id: 553ca7bd-925a-4dc5-a928-563b520842de health: HEALTH_OK services: mon: 3 daemons, quorum ceph01,ceph02,ceph03 mgr: ceph01(active), standbys: ceph02, ceph03 mds: cephfs-1/1/1 up {0=ceph01=up:active}, 2 up:standby osd: 3 osds: 3 up, 3 in data: pools: 2 pools, 200 pgs objects: 281 objects, 1024 MB usage: 6316 MB used, 293 GB / 299 GB avail pgs: 200 active+clean After resize: $ ceph -s cluster: id: 553ca7bd-925a-4dc5-a928-563b520842de health: HEALTH_OK services: mon: 3 daemons, quorum ceph01,ceph02,ceph03 mgr: ceph01(active), standbys: ceph02, ceph03 mds: cephfs-1/1/1 up {0=ceph02=up:active}, 2 up:standby osd: 3 osds: 3 up, 3 in data: pools: 2 pools, 200 pgs objects: 283 objects, 1024 MB usage: 156 GB used, 293 GB / 449 GB avail pgs: 200 active+clean So, after "growing" all OSDs by 50 GB (and object size remaining the same), the new 50 GB of additional space shows up as as used space. Also, the pool max available size stays the same. $ ceph df GLOBAL: SIZE AVAIL RAW USED %RAW USED 449G 293G 156G 34.70 POOLS: NAME ID USED %USED MAX AVAIL OBJECTS cephfs_data 2 1024M 1.09 92610M 261 cephfs_metadata 3 681k 0 92610M 22 I've tried searching around on the Internet and looked through documentation to see if/how growing bluestore volume OSDs is possible and haven't come up with anything. I'd greatly appreciate any help in this area if anyone has experience. Thanks.
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com