Re: iSCSI vs. SMB with ZFS.
On Mon, Dec 17, 2012 at 05:22:50PM -0500, Rick Macklem wrote:
> Zaphod Beeblebrox wrote:
> > Does windows 7 support nfs v4, then? Is it expected (ie: is it
> > worthwhile
> > trying) that nfsv4 would perform at a similar speed to iSCSI? It would
> > seem that this at least requires active directory (or this user name
> > mapping ... which I remember being hard).
>
> As far as I know, there is no NFSv4 in Windows. I only made the comment
> (which I admit was a bit off topic), because the previous post had stated
> "SMB or NFS, they're the same" or something like that.)
>
> There was work on an NFSv4 client for Windows being done by CITI at the
> Univ. of Michigan funded by Microsoft research, but I have no idea if it
> was ever released.
There appears to be an implementation of NFSV4 {client,server} for
Windows available from OpenText (via their acquisition of Hummingbird). This
would not be a free product. I have no experience with their NFSV4 stuff,
so have no comments on the speed...
-Fred Whiteside
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
you cannot compare file serving and block device serving. On Mon, 17 Dec 2012, Zaphod Beeblebrox wrote: Does windows 7 support nfs v4, then? Is it expected (ie: is it worthwhile trying) that nfsv4 would perform at a similar speed to iSCSI? It would seem that this at least requires active directory (or this user name mapping ... which I remember being hard). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
On Mon, Dec 17, 2012 at 2:22 PM, Rick Macklem wrote: > Zaphod Beeblebrox wrote: > > Does windows 7 support nfs v4, then? Is it expected (ie: is it > > worthwhile > > trying) that nfsv4 would perform at a similar speed to iSCSI? It would > > seem that this at least requires active directory (or this user name > > mapping ... which I remember being hard). > > As far as I know, there is no NFSv4 in Windows. I only made the comment > (which I admit was a bit off topic), because the previous post had stated > "SMB or NFS, they're the same" or something like that.) > > There was work on an NFSv4 client for Windows being done by CITI at the > Univ. of Michigan funded by Microsoft research, but I have no idea if it > was ever released. > > rick > > > > > http://www.citi.umich.edu/projects/nfsv4/ Projects: NFS Version 4 Open Source Reference Implementation We are developing an implementation of NFSv4 and NFSv4.1 for Linux http://www.citi.umich.edu/projects/nfsv4/windows/ http://www.citi.umich.edu/projects/nfsv4/windows/readme.html http://www.citi.umich.edu/projects/ Thank you very much . Mehmet Erol Sanliturk ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
Zaphod Beeblebrox wrote: > Does windows 7 support nfs v4, then? Is it expected (ie: is it > worthwhile > trying) that nfsv4 would perform at a similar speed to iSCSI? It would > seem that this at least requires active directory (or this user name > mapping ... which I remember being hard). As far as I know, there is no NFSv4 in Windows. I only made the comment (which I admit was a bit off topic), because the previous post had stated "SMB or NFS, they're the same" or something like that.) There was work on an NFSv4 client for Windows being done by CITI at the Univ. of Michigan funded by Microsoft research, but I have no idea if it was ever released. rick > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
Does windows 7 support nfs v4, then? Is it expected (ie: is it worthwhile trying) that nfsv4 would perform at a similar speed to iSCSI? It would seem that this at least requires active directory (or this user name mapping ... which I remember being hard). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
Wojciech Puchar wrote: > > With a network file system (either SMB or NFS, it doesn't matter), > > you > > need to ask the server for *each* of the following situations: > > * to ask the server if a file has been changed so the client can > > use > > cached data (if the protocol supports it) > > * to ask the server if a file (or a portion of a file) has been > > locked > > by another client > > not really if there is only one user of file - then windows know this, > but > change to behaviour you described when there are more users. > > AND FINALLY the latter behaviour fails to work properly since windows > XP > (worked fine with windows 98). If you use programs that read/write > share > same files you may be sure data corruption would happen. > > you have to set > locking = yes > oplocks = no > level2 oplocks = no > > to make it work properly but even more slow!. > Btw, NFSv4 has delegations, which are essentially level2 oplocks. They can be enabled for a server if the volumes exported via NFSv4 are not being accessed locally (including Samba). For them to work, the nfscbd needs to be running on the client(s) and the clients must have IP addresses visible to the server for a callback TCP connection (no firewalls or NAT gateways). Even with delegations working, the client caching is limited to the buffer cache. I have an experimental patch that uses on-disk caching in the client for delegated files (I call it packrats), but it is not ready for production use. Now that I have the 4.1 client in place, I plan to get back to working on it. rick > > This basically means that for almost every single IO, you need to > > ask > > the server for something, which involves network traffic and > > round-trip > > delays. > Not that. The problem is that windows do not use all free memory for > caching as with local or "local" (iSCSI) disk. > > > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
With a network file system (either SMB or NFS, it doesn't matter), you need to ask the server for *each* of the following situations: * to ask the server if a file has been changed so the client can use cached data (if the protocol supports it) * to ask the server if a file (or a portion of a file) has been locked by another client not really if there is only one user of file - then windows know this, but change to behaviour you described when there are more users. AND FINALLY the latter behaviour fails to work properly since windows XP (worked fine with windows 98). If you use programs that read/write share same files you may be sure data corruption would happen. you have to set locking = yes oplocks = no level2 oplocks = no to make it work properly but even more slow!. This basically means that for almost every single IO, you need to ask the server for something, which involves network traffic and round-trip delays. Not that. The problem is that windows do not use all free memory for caching as with local or "local" (iSCSI) disk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
On 12/12/2012 17:57, Zaphod Beeblebrox wrote: > The performance of the iSCSI disk is > about the same as the local disk for some operations --- faster for > some, slower for others. The workstation has 12G of memory and it's > my perception that iSCSI is heavily cached and that this enhances it's > performance. The second launch of a game ... or the second switch > into an area (ie: loading a specific piece of geometry again) is very > fast. > The performance on the SMB share is abysmal compared to the > performance on the iSCSI share. At the very least, there seems to be > little benifit to launching the same application twice --- which is > most likely windows fault. Think about what you have there: With iSCSI you have a block device, which is seen on your workstation as a disk drive, on which it creates a "local" file system (NTFS), and does *everything* like it is using a local disk drive. This includes caching, access permission calculations, file locking, etc. With a network file system (either SMB or NFS, it doesn't matter), you need to ask the server for *each* of the following situations: * to ask the server if a file has been changed so the client can use cached data (if the protocol supports it) * to ask the server if a file (or a portion of a file) has been locked by another client This basically means that for almost every single IO, you need to ask the server for something, which involves network traffic and round-trip delays. (there are smarter network protocols, and even extensions to SMB and NFS, but they are not widely used) signature.asc Description: OpenPGP digital signature
Re: iSCSI vs. SMB with ZFS.
-Original Message- From: Zaphod Beeblebrox Sent: Wednesday, December 12, 2012 6:57 PM To: FreeBSD Hackers Subject: iSCSI vs. SMB with ZFS. > So... I have two machines. My Fileserver is a core-2-duo machine with > FreeBSD-9.1-ish ZFS, istgt and samba 3.6. My workstation is windows 7 > on an i7. Both have GigE and are connected directly via a managed > switch with jumbo packets (specifically 9016) enabled. Both are using > tagged vlan packets to the switch (if that matters at all). My experience on samba has been that it’s slow whatever one does to tweak it (probably just too linux-centric code to start with or whatever...) Just as another datapoint – do you have tried NFS yet? Win7 has NFS available as OS component, although not installed by default? -Reko ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
common to move from area to area in the game loading, unloading and reloading the same data. My test is a valid comparison of the two modes of loading the game ... from iSCSI and from SMB. i don't know how windows cache network shares (iSCSI is treated as local not network). Here is a main problem IMHO. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
On Wed, Dec 12, 2012 at 5:16 PM, Wojciech Puchar wrote: >> about the same as the local disk for some operations --- faster for >> some, slower for others. The workstation has 12G of memory and it's >> my perception that iSCSI is heavily cached and that this enhances it's > any REAL test means doing something that will not fit in cache. That's plainly not true at all on it's face. It depends on what you're testing. In this particular test, I'm looking at the performance of the components on a singular common task --- that of running a game. It's common to run a game more than once and it's common to move from area to area in the game loading, unloading and reloading the same data. My test is a valid comparison of the two modes of loading the game ... from iSCSI and from SMB. You cold criticize me for several things --- I only tested two games or I have unrealistically large and powerful hardware, but really... consider what you are testing before you pontificate on test design. And even in the case where you want to look at the "enterprise" performance of a system, knowing both the cache performance and the disk performance is better than only knowing one or other. Throughput is a combination of these features. Pure disk performance serves as a lower bound, but cache performance (especially on some of the ZFS systems people are creating these days ... with 100's of gigs of RAM) is an equally valid statistic and optimization. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
as you show your needs for unshared data for single workstation is in order of single large hard drive. reducing drive count on file server by one and connecting this one drive directly to workstation is the best solution ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: iSCSI vs. SMB with ZFS.
about the same as the local disk for some operations --- faster for some, slower for others. The workstation has 12G of memory and it's my perception that iSCSI is heavily cached and that this enhances it's any REAL test means doing something that will not fit in cache. But this is imperfect. The iSCSI disk reserves all of it's space and the files on the disk are only accessible to the computer that mounts it. it is even more imperfect. you layout one filesystem over another filesystem. not only degraded performance but you don't have parallel access to files on this "disk". The performance on the SMB share is abysmal compared to the performance on the iSCSI share. At the very least, there seems to be little benifit to launching the same application twice --- which is most likely windows fault. This is SMB protocol. sorry it is stupid. And it doesn't make real use of cache. This is how windows file sharing works. Fine if you just want to copy files. not fine if you work on them. Will SMB always have significantly less performance than iSCSI coming depends what you do but yes SMB is not efficient. i am happy with SMB as it is enough to store users or shared documents. And it is quite fast on large file copy etc. but terrible on randomly accessing files. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
iSCSI vs. SMB with ZFS.
So... I have two machines. My Fileserver is a core-2-duo machine with FreeBSD-9.1-ish ZFS, istgt and samba 3.6. My workstation is windows 7 on an i7. Both have GigE and are connected directly via a managed switch with jumbo packets (specifically 9016) enabled. Both are using tagged vlan packets to the switch (if that matters at all). Some time ago, I created a 2T iSCSI disk on ZFS to serve the "Steam" directory (games) on my C drive as it was growing rather large. I've been quite happy with this. The performance of the iSCSI disk is about the same as the local disk for some operations --- faster for some, slower for others. The workstation has 12G of memory and it's my perception that iSCSI is heavily cached and that this enhances it's performance. The second launch of a game ... or the second switch into an area (ie: loading a specific piece of geometry again) is very fast. But this is imperfect. The iSCSI disk reserves all of it's space and the files on the disk are only accessible to the computer that mounts it. The most recent Steam update supported an easy way to put steam folders on other disks and partitions. I created another Steam folder on an SMB share from the same server and proceeded to move one of my games there. The performance on the SMB share is abysmal compared to the performance on the iSCSI share. At the very least, there seems to be little benifit to launching the same application twice --- which is most likely windows fault. I haven't done any major amount of tuning on the SMB share lately, but the last time I cared, it was setup reasonably... with TCPNODELAY and whatnot. I also notice that my copy of smbd runs with 1 thread (according to top) rather than the 11 threads that istgt uses. Does this breakdown of performance square with other's experiences? Will SMB always have significantly less performance than iSCSI coming from ZFS? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
