Re: NewStore update

2015-02-22 Thread Christoph Hellwig
On Sat, Feb 21, 2015 at 09:53:45AM -0800, Sage Weil wrote: > Ah, thanks. I guess in the buffered case though we won't block normally > anyway (unless we've hit the bdi dirty threshold). So it's probably > either aio direct or buffered write + aio fsync, depending on the cache > hints? buffered

Re: NewStore update

2015-02-21 Thread Sage Weil
On Sat, 21 Feb 2015, Christoph Hellwig wrote: > On Thu, Feb 19, 2015 at 03:50:45PM -0800, Sage Weil wrote: > > - assemble the transaction > > - start any aio writes (we could use O_DIRECT here if the new hints > > include WONTNEED?) > > Note that kernel aio only is async if you specifiy O_DIREC

Re: NewStore update

2015-02-21 Thread Christoph Hellwig
On Thu, Feb 19, 2015 at 03:50:45PM -0800, Sage Weil wrote: > - assemble the transaction > - start any aio writes (we could use O_DIRECT here if the new hints > include WONTNEED?) Note that kernel aio only is async if you specifiy O_DIRECT, otherwise io_submit will simply block. -- To unsubscrib

Re: NewStore update

2015-02-20 Thread Mark Nelson
On 02/20/2015 09:00 AM, Sage Weil wrote: On Fri, 20 Feb 2015, Haomai Wang wrote: So cool! A little notes: 1. What about sync thread in NewStore? My thought right now is that there will be a WAL thread and (maybe) a transaction commit completion thread. What do you mean by sync thread? On

Re: NewStore update

2015-02-20 Thread Haomai Wang
OK, I just viewed part of codes and realized it. It looks like we want to sync metadata each time when WAL and we ahead do_transaction jobs before WAL things. It may cause larger latency than before? Because the latency of do_transactions couldn't be simply ignore under some latency sensitive case

Re: NewStore update

2015-02-20 Thread Sage Weil
On Fri, 20 Feb 2015, Haomai Wang wrote: > So cool! > > A little notes: > > 1. What about sync thread in NewStore? My thought right now is that there will be a WAL thread and (maybe) a transaction commit completion thread. What do you mean by sync thread? One thing I want to avoid is the curre

Re: NewStore update

2015-02-20 Thread Haomai Wang
So cool! A little notes: 1. What about sync thread in NewStore? 2. Could we consider skipping WAL for large overwrite(backfill, RGW)? 3. Sorry, what means [aio_]fsync? On Fri, Feb 20, 2015 at 7:50 AM, Sage Weil wrote: > Hi everyone, > > We talked a bit about the proposed "KeyFile" backend a co

NewStore update

2015-02-19 Thread Sage Weil
Hi everyone, We talked a bit about the proposed "KeyFile" backend a couple months back. I've started putting together a basic implementation and wanted to give people and update about what things are currently looking like. We're calling it NewStore for now unless/until someone comes up with