Re: Notes from a discussion a design to allow EC overwrites

2015-12-14 Thread Samuel Just
I created a set of tickets for what I think needs to happen: http://tracker.ceph.com/issues/14031. There's a doc linked from that ticket with more details on the design. -Sam On Fri, Nov 13, 2015 at 10:26 AM, Tianshan Qu wrote: > Hi: > > sorry for half send. > > I'm

Re: Notes from a discussion a design to allow EC overwrites

2015-11-13 Thread Samuel Just
Lazily persisting the intermediate entries would certainly also work, but there's an argument that it needlessly adds to the write transaction. Actually, we probably want to avoid having small writes be full stripe writes -- with a 8+3 code the difference between modifying a single stripelet and

RE: Notes from a discussion a design to allow EC overwrites

2015-11-13 Thread Allen Samuels
This scheme fundamentally relies on the temporary objects "gracefully" transitioning into being portions of full-up long-term durable objects. This means that if the allocation size for a temporary object significantly mismatches the size of the mutation (partial stripe write) you're creating a

Re: Notes from a discussion a design to allow EC overwrites

2015-11-13 Thread Tianshan Qu
Hi: sorry for half send. I'm working on this recently. I think we should distinguish the scene of sequence write and small write. The first, overwrite is enough. The second small write's typical optimal is a log based write, which do not need ec's reconstruct and avoid parity chunk's extra

Re: Notes from a discussion a design to allow EC overwrites

2015-11-13 Thread Sage Weil
On Thu, 12 Nov 2015, Samuel Just wrote: > I was present for a discussion about allowing EC overwrites and thought it > would be good to summarize it for the list: > > Commit Protocol: > 1) client sends write to primary > 2) primary reads in partial stripes needed for partial stripe > overwrites