Also, if you’re not using Lustre 2.11 or 2.12, do not forget dnodesize=auto and recordsize=1M for OST
zfs set dnodesize=auto mdt0 zfs set dnodesize=auto ostX https://jira.whamcloud.com/browse/LU-8342 (useful for 2.10 LTS. Automatically done by Lustre for 2.11+) De : lustre-discuss <lustre-discuss-boun...@lists.lustre.org> au nom de "Carlson, Timothy S" <timothy.carl...@pnnl.gov> Date : mercredi 13 mars 2019 à 23:07 À : Riccardo Veraldi <riccardo.vera...@cnaf.infn.it>, Kurt Strosahl <stros...@jlab.org>, "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> Objet : Re: [lustre-discuss] ZFS tuning for MDT/MGS +1 on options zfs zfs_prefetch_disable=1 Might not be as critical now, but that was a must-have on Lustre 2.5.x Tim From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> On Behalf Of Riccardo Veraldi Sent: Wednesday, March 13, 2019 3:00 PM To: Kurt Strosahl <stros...@jlab.org>; lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] ZFS tuning for MDT/MGS these are the zfs settings I use on my MDSes zfs set mountpoint=none mdt0 zfs set sync=disabled mdt0 zfs set atime=off amdt0 zfs set redundant_metadata=most mdt0 zfs set xattr=sa mdt0 if youor MDT partition is on a 4KB sector disk then you can use ashift=12 when you create the filesystem but zfs is pretty smart and in my case it recognized it automatically and used ashift=12 automatically. also here are the zfs kernel modules parameters i use to ahve better performance. I use it on both MDS and OSSes options zfs zfs_prefetch_disable=1 options zfs zfs_txg_history=120 options zfs metaslab_debug_unload=1 # options zfs zfs_vdev_scheduler=deadline options zfs zfs_vdev_async_write_active_min_dirty_percent=20 # options zfs zfs_vdev_scrub_min_active=48 options zfs zfs_vdev_scrub_max_active=128 #options zfs zfs_vdev_sync_write_min_active=64 #options zfs zfs_vdev_sync_write_max_active=128 # options zfs zfs_vdev_sync_write_min_active=8 options zfs zfs_vdev_sync_write_max_active=32 options zfs zfs_vdev_sync_read_min_active=8 options zfs zfs_vdev_sync_read_max_active=32 options zfs zfs_vdev_async_read_min_active=8 options zfs zfs_vdev_async_read_max_active=32 options zfs zfs_top_maxinflight=320 options zfs zfs_txg_timeout=30 options zfs zfs_dirty_data_max_percent=40 options zfs zfs_vdev_async_write_min_active=8 options zfs zfs_vdev_async_write_max_active=32 some people may disagree with me anyway after years of trying different options I reached this stable configuration. then there are a bunch of other important Lustre level optimizations that you can do if you are looking for performance increase. Cheers Rick On 3/13/19 11:44 AM, Kurt Strosahl wrote: Good Afternoon, I'm reviewing the zfs parameters for a new metadata system and I was looking to see if anyone had examples (good or bad) of zfs parameters? I'm assuming that the MDT won't benefit from a recordsize of 1MB, and I've already set the ashift to 12. I'm using an MDT/MGS made up of a stripe across mirrored ssds. w/r, Kurt _______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
