[zfs-discuss] OpenSolaris snv_134 zfs pool hangs after some time with dedup=on
Sorry for the double post but I think this was better suite for zfs forum. I am running OpenSolaris snv_134 as a file server in a test environment, testing deduplication. I am transferring large amount of data from our production server via using rsync. The Data pool is on a separated raidz1-0 pool. The pool was running with the dedup compression turned on. The rsync would hang after a while with no more I/O going to the raidz pool (zpool iostat) and without any error message. the rpool is on it's own disk is not affected. However access to the data pool will get slower and eventually hang command such as df, du, zpool status , zfs list. System also can not be powered off from console. It will hang while closing zfs pool and require hard reset. I have now turned the dedup off on the pools and the rsync seem to be going further than before. Is this a known bug? Is there an workaround for this without rebooting the system? I am not an Solaris expert and I haven't worked on Solaris since 5.8. I would be happy to provide additional information if needed. Thanks. p.s. Since I turn off the dedup, I have finished two additional volume sync without issue. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] OpenSolaris snv_134 zfs pool hangs after some time with dedup=on
On Wed, Apr 28, 2010 at 1:51 PM, Jim Horng jho...@stretchinc.com wrote: I have now turned the dedup off on the pools and the rsync seem to be going further than before. Is this a known bug? Is there an workaround for this without rebooting the system? I am not an Solaris expert and I haven't worked on Solaris since 5.8. I would be happy to provide additional information if needed. Dedup can use a lot of RAM. If your host has limited memory, you will eventually hit a brick wall on performance. Adding more memory or a SSD L2ARC may help. -B -- Brandon High : bh...@freaks.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] OpenSolaris snv_134 zfs pool hangs after some time with dedup=on
This is not a performance issue. The rsync will hang hard and one of the child process can not be killed (I assume it's the one running on the zfs). the command gets slower I am referring to the output of the file system commands (zpool, zfs, df, du, etc) from the different shell. I left the system running idle for couple of days when it first happen and none of hanging process return. The server had 16 GB of RAM and the data pool is only 6x 1TB hard drive in one raidz vdev. (show up as a 4.4 TB zfs pool). What's the size of the L2ARC needed for dedup a 4.4TB volume? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] OpenSolaris snv_134 zfs pool hangs after some time with dedup=on
On Wed, Apr 28, 2010 at 5:09 PM, Jim Horng jho...@stretchinc.com wrote: This is not a performance issue. The rsync will hang hard and one of the child process can not be killed (I assume it's I've seen a similar issue on a b133 host that has a large DDT, but I haven't waited very long to see if it completes. You can't kill it while it's blocking on io, but once that finishes it'll die. What's the size of the L2ARC needed for dedup a 4.4TB volume? I'm not sure if there's a guideline other than big. It depends on the size of the DDT. Lots of unique data will give you a very large DDT, while 100,000 copies of the same file will give you a much smaller one. -B -- Brandon High : bh...@freaks.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
