Reviewed by: Serapheim Dimitropoulos <[email protected]> Reviewed by: Brad Lewis <[email protected]> Reviewed by: George Wilson <[email protected]>
ZFS starts syncing a TXG if there's 64MB of dirty data. This can result in very short TXG syncs, which can be inefficient because the ratio of metadata to user data is poor. Typically, this is not a real performance problem, because it was thought to only happen under very light workloads. As the (write) workload increases, the TXG sync time will increase, leading to efficiency. However, when the workload is almost entirely sync writes, especially those that use dmu_sync() to write the user data to its final resting place from open context, we can have very short TXG's even under moderate workloads, because spa_sync() isn't writing the actual user data. The problem is not the short txg's per se, but rather the high frequency of pushing out txg's, caused by the low 64MB trigger for starting a sync. Conceptually, for a given workload, each TXG is going to have a fixed amount of overhead (in terms of MB, IOPS, or time taken to write it) that is only loosely coupled to the amount of dirty data (i.e. frequency of TXG sync). Therefore, decreasing TXG sync frequency will decrease the overhead per unit time. The solution is to increase the amount of dirty data allowed before pushing out a TXG, from 64MB to 20% of zfs_dirty_data_max (820MB if total RAM >= 40GB). When we did this on the customer system in ESCL-658, overall write inflation by bandwidth decreased from ~5.5x to 3x. Upstream bug: DLPX-56162 You can view, comment on, or merge this pull request online at: https://github.com/openzfs/openzfs/pull/655 -- Commit Summary -- * 9617 too-frequent TXG sync causes excessive write inflation -- File Changes -- M usr/src/uts/common/fs/zfs/dsl_pool.c (10) M usr/src/uts/common/fs/zfs/sys/dsl_pool.h (2) M usr/src/uts/common/fs/zfs/txg.c (4) -- Patch Links -- https://github.com/openzfs/openzfs/pull/655.patch https://github.com/openzfs/openzfs/pull/655.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/655 ------------------------------------------ openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/T9f19347b7109d5d6-M0eeaed9a7a60dec400d79d3f Delivery options: https://openzfs.topicbox.com/groups
