Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Yi Zhang
On Mon, Feb 7, 2011 at 12:25 AM, Richard Elling richard.ell...@gmail.com wrote: On Feb 5, 2011, at 8:10 AM, Yi Zhang wrote: Hi all, I'm trying to achieve the same effect of UFS directio on ZFS and here is what I did: Solaris UFS directio has three functions:        1. improved async code

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Roch
Le 7 févr. 2011 à 06:25, Richard Elling a écrit : On Feb 5, 2011, at 8:10 AM, Yi Zhang wrote: Hi all, I'm trying to achieve the same effect of UFS directio on ZFS and here is what I did: Solaris UFS directio has three functions: 1. improved async code path 2. multiple

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Yi Zhang
On Mon, Feb 7, 2011 at 10:26 AM, Roch roch.bourbonn...@oracle.com wrote: Le 7 févr. 2011 à 06:25, Richard Elling a écrit : On Feb 5, 2011, at 8:10 AM, Yi Zhang wrote: Hi all, I'm trying to achieve the same effect of UFS directio on ZFS and here is what I did: Solaris UFS directio has

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Brandon High
On Mon, Feb 7, 2011 at 6:15 AM, Yi Zhang yizhan...@gmail.com wrote: On Mon, Feb 7, 2011 at 12:25 AM, Richard Elling richard.ell...@gmail.com wrote: Solaris UFS directio has three functions:        1. improved async code path        2. multiple concurrent writers        3. no buffering

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Yi Zhang
On Mon, Feb 7, 2011 at 1:06 PM, Brandon High bh...@freaks.com wrote: On Mon, Feb 7, 2011 at 6:15 AM, Yi Zhang yizhan...@gmail.com wrote: On Mon, Feb 7, 2011 at 12:25 AM, Richard Elling richard.ell...@gmail.com wrote: Solaris UFS directio has three functions:        1. improved async code path

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Brandon High
On Mon, Feb 7, 2011 at 10:29 AM, Yi Zhang yizhan...@gmail.com wrote: I already set primarycache to metadata, and I'm not concerned about caching reads, but caching writes. It appears writes are indeed cached judging from the time of 2.a) compared to UFS+directio. More specifically,

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Yi Zhang
On Mon, Feb 7, 2011 at 1:51 PM, Brandon High bh...@freaks.com wrote: On Mon, Feb 7, 2011 at 10:29 AM, Yi Zhang yizhan...@gmail.com wrote: I already set primarycache to metadata, and I'm not concerned about caching reads, but caching writes. It appears writes are indeed cached judging from the

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Roy Sigurd Karlsbakk
Maybe I didn't make my intention clear. UFS with directio is reasonably close to a raw disk from my application's perspective: when the app writes to a file location, no buffering happens. My goal is to find a way to duplicate this on ZFS. There really is no need to do this on ZFS. Using an

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Yi Zhang
On Mon, Feb 7, 2011 at 2:21 PM, Brandon High bh...@freaks.com wrote: On Mon, Feb 7, 2011 at 11:17 AM, Yi Zhang yizhan...@gmail.com wrote: Maybe I didn't make my intention clear. UFS with directio is reasonably close to a raw disk from my application's perspective: when the app writes to a file

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Nico Williams
On Mon, Feb 7, 2011 at 1:17 PM, Yi Zhang yizhan...@gmail.com wrote: On Mon, Feb 7, 2011 at 1:51 PM, Brandon High bh...@freaks.com wrote: Maybe I didn't make my intention clear. UFS with directio is reasonably close to a raw disk from my application's perspective: when the app writes to a file

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Yi Zhang
On Mon, Feb 7, 2011 at 2:42 PM, Nico Williams n...@cryptonector.com wrote: On Mon, Feb 7, 2011 at 1:17 PM, Yi Zhang yizhan...@gmail.com wrote: On Mon, Feb 7, 2011 at 1:51 PM, Brandon High bh...@freaks.com wrote: Maybe I didn't make my intention clear. UFS with directio is reasonably close to a

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Roch
Le 7 févr. 2011 à 17:08, Yi Zhang a écrit : On Mon, Feb 7, 2011 at 10:26 AM, Roch roch.bourbonn...@oracle.com wrote: Le 7 févr. 2011 à 06:25, Richard Elling a écrit : On Feb 5, 2011, at 8:10 AM, Yi Zhang wrote: Hi all, I'm trying to achieve the same effect of UFS directio on ZFS and

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Bill Sommerfeld
On 02/07/11 11:49, Yi Zhang wrote: The reason why I tried that is to get the side effect of no buffering, which is my ultimate goal. ultimate = final. you must have a goal beyond the elimination of buffering in the filesystem. if the writes are made durable by zfs when you need them to be

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Yi Zhang
On Mon, Feb 7, 2011 at 2:54 PM, Nico Williams n...@cryptonector.com wrote: On Mon, Feb 7, 2011 at 1:49 PM, Yi Zhang yizhan...@gmail.com wrote: Please see my previous email for a high-level discussion of my application. I know that I don't really need O_DSYNC. The reason why I tried that is to

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Yi Zhang
On Mon, Feb 7, 2011 at 3:14 PM, Bill Sommerfeld sommerf...@alum.mit.edu wrote: On 02/07/11 11:49, Yi Zhang wrote: The reason why I tried that is to get the side effect of no buffering, which is my ultimate goal. ultimate = final.  you must have a goal beyond the elimination of buffering in

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread David Dyer-Bennet
On Mon, February 7, 2011 14:49, Yi Zhang wrote: On Mon, Feb 7, 2011 at 3:14 PM, Bill Sommerfeld sommerf...@alum.mit.edu wrote: On 02/07/11 11:49, Yi Zhang wrote: The reason why I tried that is to get the side effect of no buffering, which is my ultimate goal. ultimate = final.  you must

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Yi Zhang
On Mon, Feb 7, 2011 at 3:47 PM, Nico Williams n...@cryptonector.com wrote: On Mon, Feb 7, 2011 at 2:39 PM, Yi Zhang yizhan...@gmail.com wrote: On Mon, Feb 7, 2011 at 2:54 PM, Nico Williams n...@cryptonector.com wrote: ZFS cannot not buffer.  The reason is that ZFS likes to batch transactions

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Bill Sommerfeld
On 02/07/11 12:49, Yi Zhang wrote: If buffering is on, the running time of my app doesn't reflect the actual I/O cost. My goal is to accurately measure the time of I/O. With buffering on, ZFS would batch up a bunch of writes and change both the original I/O activity and the time. if batching

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Erik Trimble
On 2/7/2011 1:10 PM, Yi Zhang wrote: [snip] This is actually what I did for 2.a) in my original post. My concern there is that ZFS' internal write buffering makes it hard to get a grip on my application's behavior. I want to present my application's raw I/O performance without too much outside

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-07 Thread Richard Elling
On Feb 7, 2011, at 1:10 PM, Yi Zhang wrote: This is actually what I did for 2.a) in my original post. My concern there is that ZFS' internal write buffering makes it hard to get a grip on my application's behavior. I want to present my application's raw I/O performance without too much

Re: [zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-06 Thread Richard Elling
On Feb 5, 2011, at 8:10 AM, Yi Zhang wrote: Hi all, I'm trying to achieve the same effect of UFS directio on ZFS and here is what I did: Solaris UFS directio has three functions: 1. improved async code path 2. multiple concurrent writers 3. no buffering Of the

[zfs-discuss] Understanding directio, O_DSYNC and zfs_nocacheflush on ZFS

2011-02-05 Thread Yi Zhang
Hi all, I'm trying to achieve the same effect of UFS directio on ZFS and here is what I did: 1. Set the primarycache of zfs to metadata and secondarycache to none, recordsize to 8K (to match the unit size of writes) 2. Run my test program (code below) with different options and measure the