Hi All, I've run into a problem with my OpenSolaris system and NTFS, I can't seem to make sense of it.
The server running the virtual machines is Ubuntu Server 10.04 running KVM. Storage is presented via NFS over Infiniband. ZFS is not running compression or dedup. Zil is also currently disabled because it was causing terrible NFS performance. ESXi also displayed the same behaviour when running Windows VM's. OpenSolaris box is running - OS b147 Xeon 5520 24GB DDR3 24x 1.5TB 7200rpm sata drives 24 bay expander connected to an LSI raid card via 4 lane SAS cable 2x 64GB Intel X25-M L2ARC 2x 32GB Intel X25-M SLOG Host - 4KB reads & writes are full speed. Tested using DD. Also tested using Samba writing to the NFS share (Bare metal PC -> samba -> NFS) and it was writing as fast as the server could dish out the data (50MB/s). Test VM 1 - OS is Ubuntu Server 10.10 on ext4. Reads & writes are full speed at 4kb, tested using DD. Sequential read/write seems fine. Tested using Samba also, same results as above (50MB/s). Test VM 2- OS is Windows Server 2008 on NTFS 4kb cluster size. 4KB reads & writes are 30MB/s, tested using the ATTO disk benchmark. Sequential reads and writes max out at 30MB/s also and generate a lot of IO on the storage box. IOStat on the storage box shows 3k+ IOPS and only 30MB/s throughput. Also tested a bare metal system running iSCSI over gigabit. NTFS (default 4kb cluster size) directly on the ZFS block volume. Results are the same as Test VM 2, lots of small IO (3k+ IOPS) and small throughput 30MB/s I also did the same test using SRP. 4kb reads and writes have the same 30MB/s. Any IO above or equal to 128kb seems to pick up speed drastically, easily 700MB/s. I have a feeling it's to do with ZFS's recordsize property but haven't been able to find any solid testing done with NTFS. I'm going to do some testing using smaller record sizes tonight to see if that helps the issue. At the moment I'm surviving on cache and am quickly running out of capacity. Can anyone suggest any further tests or have any idea about what's going on? Thanks -Matt _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
