This is CentOS 7.3 ZoL version 0.6.5.9-1 [root@clei22 ~]# lsscsi
[2:0:0:0] disk ATA INTEL SSDSC2CW24 400i /dev/sda [3:0:0:0] disk ATA HGST HUS724040AL AA70 /dev/sdb [4:0:0:0] disk ATA WDC WD2002FYPS-0 1G01 /dev/sdc [root@clei22 ~]# pvs ;vgs;lvs PV VG Fmt Attr PSize PFree /dev/mapper/INTEL_SSDSC2CW240A3_CVCV306302RP240CGN vg_cache lvm2 a-- 223.57g 0 /dev/sdc2 centos_clei22 lvm2 a-- 1.82t 64.00m VG #PV #LV #SN Attr VSize VFree centos_clei22 1 3 0 wz--n- 1.82t 64.00m vg_cache 1 2 0 wz--n- 223.57g 0 LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert home centos_clei22 -wi-ao---- 1.74t root centos_clei22 -wi-ao---- 50.00g swap centos_clei22 -wi-ao---- 31.44g lv_cache vg_cache -wi-ao---- 213.57g lv_slog vg_cache -wi-ao---- 10.00g [root@clei22 ~]# zpool status -v pool: zclei22 state: ONLINE scan: scrub repaired 0 in 0h0m with 0 errors on Tue Feb 28 14:16:07 2017 config: NAME STATE READ WRITE CKSUM zclei22 ONLINE 0 0 0 HGST_HUS724040ALA640_PN2334PBJ4SV6T1 ONLINE 0 0 0 logs lv_slog ONLINE 0 0 0 cache lv_cache ONLINE 0 0 0 errors: No known data errors *ZFS config:* [root@clei22 ~]# zfs get all zclei22/01 NAME PROPERTY VALUE SOURCE zclei22/01 type filesystem - zclei22/01 creation Tue Feb 28 14:06 2017 - zclei22/01 used 389G - zclei22/01 available 3.13T - zclei22/01 referenced 389G - zclei22/01 compressratio 1.01x - zclei22/01 mounted yes - zclei22/01 quota none default zclei22/01 reservation none default zclei22/01 recordsize 128K local zclei22/01 mountpoint /zclei22/01 default zclei22/01 sharenfs off default zclei22/01 checksum on default zclei22/01 compression off local zclei22/01 atime on default zclei22/01 devices on default zclei22/01 exec on default zclei22/01 setuid on default zclei22/01 readonly off default zclei22/01 zoned off default zclei22/01 snapdir hidden default zclei22/01 aclinherit restricted default zclei22/01 canmount on default zclei22/01 xattr sa local zclei22/01 copies 1 default zclei22/01 version 5 - zclei22/01 utf8only off - zclei22/01 normalization none - zclei22/01 casesensitivity sensitive - zclei22/01 vscan off default zclei22/01 nbmand off default zclei22/01 sharesmb off default zclei22/01 refquota none default zclei22/01 refreservation none default zclei22/01 primarycache metadata local zclei22/01 secondarycache metadata local zclei22/01 usedbysnapshots 0 - zclei22/01 usedbydataset 389G - zclei22/01 usedbychildren 0 - zclei22/01 usedbyrefreservation 0 - zclei22/01 logbias latency default zclei22/01 dedup off default zclei22/01 mlslabel none default zclei22/01 sync disabled local zclei22/01 refcompressratio 1.01x - zclei22/01 written 389G - zclei22/01 logicalused 396G - zclei22/01 logicalreferenced 396G - zclei22/01 filesystem_limit none default zclei22/01 snapshot_limit none default zclei22/01 filesystem_count none default zclei22/01 snapshot_count none default zclei22/01 snapdev hidden default zclei22/01 acltype off default zclei22/01 context none default zclei22/01 fscontext none default zclei22/01 defcontext none default zclei22/01 rootcontext none default zclei22/01 relatime off default zclei22/01 redundant_metadata all default zclei22/01 overlay off default On Fri, Mar 3, 2017 at 2:52 PM, Juan Pablo <pablo.localh...@gmail.com> wrote: > Which operating system version are you using for your zfs storage? > do: > zfs get all your-pool-name > use arc_summary.py from freenas git repo if you wish. > > > 2017-03-03 10:33 GMT-03:00 Arman Khalatyan <arm2...@gmail.com>: > >> Pool load: >> [root@clei21 ~]# zpool iostat -v 1 >> capacity operations >> bandwidth >> pool alloc free read write >> read write >> -------------------------------------- ----- ----- ----- ----- >> ----- ----- >> zclei21 10.1G 3.62T 0 112 >> 823 8.82M >> HGST_HUS724040ALA640_PN2334PBJ52XWT1 10.1G 3.62T 0 46 >> 626 4.40M >> logs - - - - >> - - >> lv_slog 225M 9.72G 0 66 >> 198 4.45M >> cache - - - - >> - - >> lv_cache 9.81G 204G 0 46 >> 56 4.13M >> -------------------------------------- ----- ----- ----- ----- >> ----- ----- >> >> capacity operations >> bandwidth >> pool alloc free read write >> read write >> -------------------------------------- ----- ----- ----- ----- >> ----- ----- >> zclei21 10.1G 3.62T 0 191 >> 0 12.8M >> HGST_HUS724040ALA640_PN2334PBJ52XWT1 10.1G 3.62T 0 0 >> 0 0 >> logs - - - - >> - - >> lv_slog 225M 9.72G 0 191 >> 0 12.8M >> cache - - - - >> - - >> lv_cache 9.83G 204G 0 218 >> 0 20.0M >> -------------------------------------- ----- ----- ----- ----- >> ----- ----- >> >> capacity operations >> bandwidth >> pool alloc free read write >> read write >> -------------------------------------- ----- ----- ----- ----- >> ----- ----- >> zclei21 10.1G 3.62T 0 191 >> 0 12.7M >> HGST_HUS724040ALA640_PN2334PBJ52XWT1 10.1G 3.62T 0 0 >> 0 0 >> logs - - - - >> - - >> lv_slog 225M 9.72G 0 191 >> 0 12.7M >> cache - - - - >> - - >> lv_cache 9.83G 204G 0 72 >> 0 7.68M >> -------------------------------------- ----- ----- ----- ----- >> ----- ----- >> >> >> On Fri, Mar 3, 2017 at 2:32 PM, Arman Khalatyan <arm2...@gmail.com> >> wrote: >> >>> Glusterfs now in healing mode: >>> Receiver: >>> [root@clei21 ~]# arcstat.py 1 >>> time read miss miss% dmis dm% pmis pm% mmis mm% arcsz >>> c >>> 13:24:49 0 0 0 0 0 0 0 0 0 4.6G >>> 31G >>> 13:24:50 154 80 51 80 51 0 0 80 51 4.6G >>> 31G >>> 13:24:51 179 62 34 62 34 0 0 62 42 4.6G >>> 31G >>> 13:24:52 148 68 45 68 45 0 0 68 45 4.6G >>> 31G >>> 13:24:53 140 64 45 64 45 0 0 64 45 4.6G >>> 31G >>> 13:24:54 124 48 38 48 38 0 0 48 38 4.6G >>> 31G >>> 13:24:55 157 80 50 80 50 0 0 80 50 4.7G >>> 31G >>> 13:24:56 202 68 33 68 33 0 0 68 41 4.7G >>> 31G >>> 13:24:57 127 54 42 54 42 0 0 54 42 4.7G >>> 31G >>> 13:24:58 126 50 39 50 39 0 0 50 39 4.7G >>> 31G >>> 13:24:59 116 40 34 40 34 0 0 40 34 4.7G >>> 31G >>> >>> >>> Sender >>> [root@clei22 ~]# arcstat.py 1 >>> time read miss miss% dmis dm% pmis pm% mmis mm% arcsz >>> c >>> 13:28:37 8 2 25 2 25 0 0 2 25 468M >>> 31G >>> 13:28:38 1.2K 727 62 727 62 0 0 525 54 469M >>> 31G >>> 13:28:39 815 508 62 508 62 0 0 376 55 469M >>> 31G >>> 13:28:40 994 624 62 624 62 0 0 450 54 469M >>> 31G >>> 13:28:41 783 456 58 456 58 0 0 338 50 470M >>> 31G >>> 13:28:42 916 541 59 541 59 0 0 390 50 470M >>> 31G >>> 13:28:43 768 437 56 437 57 0 0 313 48 471M >>> 31G >>> 13:28:44 877 534 60 534 60 0 0 393 53 470M >>> 31G >>> 13:28:45 957 630 65 630 65 0 0 450 57 470M >>> 31G >>> 13:28:46 819 479 58 479 58 0 0 357 51 471M >>> 31G >>> >>> >>> On Thu, Mar 2, 2017 at 7:18 PM, Juan Pablo <pablo.localh...@gmail.com> >>> wrote: >>> >>>> hey, >>>> what are you using for zfs? get an arc status and show please >>>> >>>> >>>> 2017-03-02 9:57 GMT-03:00 Arman Khalatyan <arm2...@gmail.com>: >>>> >>>>> no, >>>>> ZFS itself is not on top of lvm. only ssd was spitted by lvm for >>>>> slog(10G) and cache (the rest) >>>>> but in any-case the ssd does not help much on glusterfs/ovirt load it >>>>> has almost 100% cache misses....:( (terrible performance compare with nfs) >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, Mar 2, 2017 at 1:47 PM, FERNANDO FREDIANI < >>>>> fernando.fredi...@upx.com> wrote: >>>>> >>>>>> Am I understanding correctly, but you have Gluster on the top of ZFS >>>>>> which is on the top of LVM ? If so, why the usage of LVM was necessary ? >>>>>> I >>>>>> have ZFS with any need of LVM. >>>>>> >>>>>> Fernando >>>>>> >>>>>> On 02/03/2017 06:19, Arman Khalatyan wrote: >>>>>> >>>>>> Hi, >>>>>> I use 3 nodes with zfs and glusterfs. >>>>>> Are there any suggestions to optimize it? >>>>>> >>>>>> host zfs config 4TB-HDD+250GB-SSD: >>>>>> [root@clei22 ~]# zpool status >>>>>> pool: zclei22 >>>>>> state: ONLINE >>>>>> scan: scrub repaired 0 in 0h0m with 0 errors on Tue Feb 28 14:16:07 >>>>>> 2017 >>>>>> config: >>>>>> >>>>>> NAME STATE READ WRITE >>>>>> CKSUM >>>>>> zclei22 ONLINE 0 0 >>>>>> 0 >>>>>> HGST_HUS724040ALA640_PN2334PBJ4SV6T1 ONLINE 0 0 >>>>>> 0 >>>>>> logs >>>>>> lv_slog ONLINE 0 0 >>>>>> 0 >>>>>> cache >>>>>> lv_cache ONLINE 0 0 >>>>>> 0 >>>>>> >>>>>> errors: No known data errors >>>>>> >>>>>> Name: >>>>>> GluReplica >>>>>> Volume ID: >>>>>> ee686dfe-203a-4caa-a691-26353460cc48 >>>>>> Volume Type: >>>>>> Replicate (Arbiter) >>>>>> Replica Count: >>>>>> 2 + 1 >>>>>> Number of Bricks: >>>>>> 3 >>>>>> Transport Types: >>>>>> TCP, RDMA >>>>>> Maximum no of snapshots: >>>>>> 256 >>>>>> Capacity: >>>>>> 3.51 TiB total, 190.56 GiB used, 3.33 TiB free >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Users mailing >>>>>> listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Users mailing list >>>>>> Users@ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Users mailing list >>>>> Users@ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>> >>>>> >>>> >>> >> >
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users