On Tue, Aug 4, 2009 at 9:57 AM, Charles Baker<no-re...@opensolaris.org> wrote: >> My testing has shown some serious problems with the >> iSCSI implementation for OpenSolaris. >> >> I setup a VMware vSphere 4 box with RAID 10 >> direct-attached storage and 3 virtual machines: >> - OpenSolaris 2009.06 (snv_111b) running 64-bit >> - CentOS 5.3 x64 (ran yum update) >> - Ubuntu Server 9.04 x64 (ran apt-get upgrade) >> >> I gave each virtual 2 GB of RAM, a 32 GB drive and >> setup a 16 GB iSCSI target on each (the two Linux vms >> used iSCSI Enterprise Target 0.4.16 with blockio). >> VMware Tools was installed on each. No tuning was >> done on any of the operating systems. >> >> I ran two tests for write performance - one one the >> server itself and one from my Mac connected via >> Gigabit (mtu of 1500) iSCSI connection using >> globalSAN’s latest initiator. >> >> Here’s what I used on the servers: >> time dd if=/dev/zero of=/root/testfile bs=1048576k >> count=4 >> and the Mac OS with the iSCSI connected drive >> (formatted with GPT / Mac OS Extended journaled): >> time dd if=/dev/zero of=/Volumes/test/testfile >> bs=1048576k count=4 >> >> The results were very interesting (all calculations >> using 1 MB = 1,084,756 bytes) >> >> For OpenSolaris, the local write performance averaged >> 86 MB/s. I turned on lzjb compression for rpool (zfs >> set compression=lzjb rpool) and it went up to 414 >> MB/s since I’m writing zeros). The average >> performance via iSCSI was an abysmal 16 MB/s (even >> with compression turned on - with it off, 13 MB/s). >> >> For CentOS (ext3), local write performance averaged >> 141 MB/s. iSCSI performance was 78 MB/s (almost as >> fast as local ZFS performance on the OpenSolaris >> server when compression was turned off). >> >> Ubuntu Server (ext4) had 150 MB/s for the local >> write. iSCSI performance averaged 80 MB/s. >> >> One of the main differences between the three virtual >> machines was that the iSCSI target on the Linux >> machines used partitions with no file system. On >> OpenSolaris, the iSCSI target created sits on top of >> ZFS. That creates a lot of overhead (although you do >> get some great features). >> >> Since all the virtual machines were connected to the >> same switch (with the same MTU), had the same amount >> of RAM, used default configurations for the operating >> systems, and sat on the same RAID 10 storage, I’d say >> it was a pretty level playing field. >> >> While jumbo frames will help iSCSI performance, it >> won’t overcome inherit limitations of the iSCSI >> target’s implementation.
If you want to host your VMs from Solaris (Open or not) use NFS right now as the iSCSI implementation is still quite a bit immature and won't perform nearly as good as the Linux implementation. Until comstar stabilizes and replaces iscsitgt I would hold off on iSCSI on Solaris. -Ross _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
