Ugh! If you received a direct response to me instead of via the list, apologies for that.
Rob: I'm just reporting the news. The RFE is out there. Just like SLOGs, I happen to think it a good idea, personally, but that's my personal opinion. If it makes dedup more usable, I don't see the harm. Taemun: The issue, as I understand it, is not "use-lots-of-cpu" or "just dies from paging". I believe it is more to do with all of the small, random reads/writes in updating the DDT. Remember, the DDT is stored within the pool, just as the ZIL is if you don't have a SLOG. (The S in SLOG standing for "separate".) So all the DDT updates are in competition for I/O with the actual data deletion. If the DDT could be stored as a separate VDEV already, I'm sure a way would have been hacked together by someone (likely someone on this list). Hence, the need for the RFE to create this functionality where it does not currently exist. The DDT is separate from the ARC or L2ARC. Here's the bug: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6913566 If I'm incorrect, someone please let me know. Markus: Yes, the issue would appear to be dataset size vs. RAM size. Sounds like an area ripe for testing, much like RAID Z3 performance. Cheers all! On Tue, Feb 16, 2010 at 00:20, taemun <tae...@gmail.com> wrote: > The system in question has 8GB of ram. It never paged during the > import (unless I was asleep at that point, but anyway). > > It ran for 52 hours, then started doing 47% kernel cpu usage. At this > stage, dtrace stopped responding, and so iopattern died, as did > iostat. It was also increasing ram usage rapidly (15mb / minute). > After an hour of that, the cpu went up to 76%. An hour later, CPU > usage stopped. Hard drives were churning throughout all of this > (albeit at a rate that looks like each vdev is being controller by a > single threaded operation). > > I'm guessing that if you don't have enough ram, it gets stuck on the > use-lots-of-cpu phase, and just dies from too much paging. Of course, > I have absolutely nothing to back that up. > > Personally, I think that if L2ARC devices were persistent, we already > have the mechanism in place for storing the DDT as a "seperate vdev". > The problem is, there is nothing you can run at boot time to populate > the L2ARC, so the dedup writes are ridiculously slow until the cache > is warm. If the cache stayed warm, or there was an option to forcibly > warm up the cache, this could be somewhat alleviated. > > Cheers > -- "You can choose your friends, you can choose the deals." - Equity Private "If Linux is faster, it's a Solaris bug." - Phil Harman Blog - http://whatderass.blogspot.com/ Twitter - @khyron4eva
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
