Concurrency Question

2013-01-29 Thread Ian Boston
Hi, Are the concurrency characteristics of MongoDB [1] relevant to write concurrency in Oak ? IIUC from the presentation MongoDB pre 2.2 had global exclusive write locks and post 2.2 has per database exclusive write locks which might be scoped to the shard if the update only impacts a single shard

Re: Conflict handling through rebasing branches

2013-01-29 Thread Michael Dürig
I added a Wiki page [1] summarising and amending this proposal. Michael [1] http://wiki.apache.org/jackrabbit/Conflict%20handling%20through%20rebasing%20branches On 17.1.13 17:15, Michael Dürig wrote: Hi, There are various places in Oak where conflicting updates to the content tree can oc

Re: MongoMK^2 design proposal

2013-01-29 Thread Tyson Norris
On Jan 29, 2013, at 3:45 AM, "Jukka Zitting" wrote: > Hi, > > On Tue, Jan 29, 2013 at 1:21 PM, Thomas Mueller wrote: >> It's not clear to me how to support scalable concurrent writes. This is >> also a problem with the current MongoMK design, but I in your design I >> actually see more proble

Re: MongoMK^2 design proposal

2013-01-29 Thread Jukka Zitting
Hi, On Tue, Jan 29, 2013 at 1:43 PM, Michael Dürig wrote: > Maybe we can even pick up an earlier idea and use the node type information > (i.e. template records here) to optimise how nodes are stored. That would clearly be useful but I'm a bit hesitant about mixing higher level concerns like nod

Re: MongoMK^2 design proposal

2013-01-29 Thread Jukka Zitting
Hi, On Tue, Jan 29, 2013 at 2:07 PM, Thomas Mueller wrote: > To better understand the design it would help me if you could list the > MongoDb collections, and the key / properties / values. I guess the > segment ids are MongoDB object ids? Or is it (part of) the path? Here's a quick draft, most

Re: MongoMK^2 design proposal

2013-01-29 Thread Thomas Mueller
Hi, To better understand the design it would help me if you could list the MongoDb collections, and the key / properties / values. I guess the segment ids are MongoDB object ids? Or is it (part of) the path? > Segments are immutable, so a commit would create a new segment So there are no MongoDB

Re: MongoMK^2 design proposal

2013-01-29 Thread Jukka Zitting
Hi, On Tue, Jan 29, 2013 at 1:21 PM, Thomas Mueller wrote: > It's not clear to me how to support scalable concurrent writes. This is > also a problem with the current MongoMK design, but I in your design I > actually see more problems in this area (concurrent writes to nodes in the > same segment

Re: MongoMK^2 design proposal

2013-01-29 Thread Michael Dürig
Hi, Thanks for putting this together. I think this makes a lot of sense. Primarily since it will reduce coupling with the underlying storage mechanism. But also since it pro-actively tackles our main pain points like write scalability, large and flat hierarchies, fragmentation and remotabili

Re: MongoMK^2 design proposal

2013-01-29 Thread Thomas Mueller
Hi, It's not clear to me how to support scalable concurrent writes. This is also a problem with the current MongoMK design, but I in your design I actually see more problems in this area (concurrent writes to nodes in the same segment for example). But maybe it's just that I don't understand this

Re: MongoMK^2 design proposal

2013-01-29 Thread Alexander Klimetschek
On 29.01.2013, at 11:01, Marcel Reutegger wrote: > What is not yet clear to me is, how do you decide what the size of a segment > should be? This can be quite delicate. On the one hand you want to avoid > small segments because they are inefficient with all the UUID lookups. But > larger segments

Re: MongoMK^2 design proposal

2013-01-29 Thread Jukka Zitting
Hi, On Tue, Jan 29, 2013 at 12:01 PM, Marcel Reutegger wrote: >> both internal storage (as seen below) and garbage collection. Segments >> that are no longer referenced can be efficiently identified by >> traversing the graph of segment-level references without having to >> parse or even fetch th

RE: MongoMK^2 design proposal

2013-01-29 Thread Marcel Reutegger
Hi, > + > MongoMK^2 design proposal > + > > Segments > > > The content tree and all its revisions are stored in a collection of > immutable *segments*. Each segment is identified by a UUID and typically > contains a continuous subset of th

Re: [Errored] apache/jackrabbit-oak#496 (trunk - 5072d8d)

2013-01-29 Thread Jukka Zitting
Hi, On Tue, Jan 29, 2013 at 9:54 AM, Marcel Reutegger wrote: > I'm sorry, my bad. I assumed our CI builds don't have access to a MongoDB. The Travis build does (that's why we have it in addition to the buildbot one), but the build time is limited since it's running on free and shared hardware. S