[ceph-users] too few pgs in cache tier
Hi all, we use an EC-Pool with an small cache tier in front of, for our archive-data (4 * 16TB VM-disks). The ec-pool has k=3;m=2 because we startet with 5 nodes and want to migrate to an new ec-pool with k=5;m=2. Therefor we migrate one VM-disk (16TB) from the ceph-cluster to an fc-raid with the proxmox-ve interface move disk. The move was finished, but during removing the ceph-vm file the warning 'ssd-archiv' at/near target max; pool ssd-archiv has too few pgs occour. Some hour later only the second warning exsist. ceph health detail HEALTH_WARN pool ssd-archiv has too few pgs pool ssd-archiv objects per pg (51196) is more than 14.7709 times cluster average (3466) info about the image, which was deleted: rbd image 'vm-409-disk-1': size 16384 GB in 4194304 objects order 22 (4096 kB objects) block_name_prefix: rbd_data.2b8fda574b0dc51 format: 2 features: layering I think we hit http://tracker.ceph.com/issues/8103 but normaly one reading should not put the data in the cache tier, or?? Is deleting a second read?? Our ceph version: 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578) Regards Udo ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] too few pgs in cache tier
On 27/02/2015, at 17.04, Udo Lembke ulem...@polarzone.de wrote: ceph health detail HEALTH_WARN pool ssd-archiv has too few pgs Slightly different I had an issue with my Ceph Cluster underneath a PVE cluster yesterday. Had two Ceph pools for RBD virt disks, vm_images (boot hdd images) + rbd_data (extra hdd images). Then while adding pools for a rados GW (.rgw.*) suddenly health status said that my vm_images pool had too few PGs, thus I ran: ceph osd pool set vm_images pg_num larger_number ceph osd pool set vm_images pgp_num larger_number Kicking off a 20 min rebalancing with a lot of IO in the Ceph Cluster, eventually Ceph Cluster was fine again, only almost all my PVE VMs ended up in stopped state, wondering why, a watchdog thingy maybe... /Steffen signature.asc Description: Message signed with OpenPGP using GPGMail ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
