Recently I noticed that on one of our systems l2arc_feed_thread was consuming non insignificant amount of CPU time. I looked at some stats (FreeBSD has local changes that add a few kstats for L2ARC) and it seems that the thread was busy scanning and re-scanning buffers that are already in L2 ARC. Buffers were being skipped because l2arc_write_eligible was returning false because of ab->b_l2hdr != NULL. As far as I can see the thread was doing a pass per second and it typically scanned about 2 million buffers per pass. Typically walking over a buffer list was aborted due to passed_sz > headroom.
The system in question has a quite large ARC with maximum size of 60GB. 26GB were actually in use and it seems that most of the buffers were rather small, hash_elements == 3634055. Perhaps, there could be some optimization to avoid pointless walking over millions of buffers in situations like this? P.S. Because of another local FreeBSD change the feed thread was scanning about 16 times more buffers than it would on illumos, so the issue was more prominent with the thread consuming about 40% of a core. -- Andriy Gapon
_______________________________________________ developer mailing list developer@open-zfs.org http://lists.open-zfs.org/mailman/listinfo/developer
