Oak JCR Observation scalability aspects and concerns

2013-10-21 Thread Chetan Mehrotra
Below are the details about discussion held [0] so far on factors which possibly hinder scaling JCR Observation. It also touches upon area where usage pattern of applications running on JCR (like Sling) effects performance Concerns - Marcel - Basic concern raised is that listeners wit

Re: Detecting move operations in node state diffs

2013-10-21 Thread Michael Dürig
Hi, I implemented a very rough POC of the algorithm outlined below. See [1] for the implementation itself. On move a node is annotated with its source path in NodeBuilder.moveTo(). Later moves can be extracted through the standalone MoveDetector class. See MoveDetectorTest for details. MoveD

jackrabbit-oak build #2420: Broken

2013-10-21 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #2420 Status: Broken Duration: 746 seconds Commit: 9c28da397b7a07ea21f5a039a894313638933cf6 (trunk) Author: Thomas Mueller Message: OAK-1083 Query with descendent node and access control fails to return result g

Re: Oak JCR Observation scalability aspects and concerns

2013-10-21 Thread Jukka Zitting
Hi, On Mon, Oct 21, 2013 at 6:38 AM, Chetan Mehrotra wrote: > Marcel - Basic concern raised is that listeners without any filter > would cause lots of reads on the repository. these kind of listeners > would pull in modifications of all sessions performing distributed > writes. In our view this w

Re: Oak JCR Observation scalability aspects and concerns

2013-10-21 Thread Bertrand Delacretaz
On Mon, Oct 21, 2013 at 3:17 PM, Jukka Zitting wrote: > ...Instead of an repository problem (like diffing, event creation, etc.), > this analysis tells me that the bottleneck here is the application > that tries to listen to so many events... FWIW, by default Sling does have a listener that catch

Re: Oak JCR Observation scalability aspects and concerns

2013-10-21 Thread Jukka Zitting
Hi, On Mon, Oct 21, 2013 at 9:47 AM, Bertrand Delacretaz wrote: > On Mon, Oct 21, 2013 at 3:17 PM, Jukka Zitting > wrote: >> ...Instead of an repository problem (like diffing, event creation, etc.), >> this analysis tells me that the bottleneck here is the application >> that tries to listen to

Re: jackrabbit-oak build #2420: Broken

2013-10-21 Thread Thomas Mueller
Out of memory in the ObservationTestŠ On 10/21/13 2:42 PM, "Travis CI" wrote: >Build Update for apache/jackrabbit-oak >- > >Build: #2420 >Status: Broken > >Duration: 746 seconds >Commit: 9c28da397b7a07ea21f5a039a894313638933cf6 (trunk) >Author: Thomas Muelle

Re: Detecting move operations in node state diffs

2013-10-21 Thread Thomas Mueller
Hi, > extra pass On how to avoid this extra pass. Not strictly backward compatible, but I wonder how much it would break: what if observation would deliver two events for moved nodes: the "node moved" event (added at the target), plus the "node deleted" event (deleted at the source)? The one use

jackrabbit-oak build #2421: Fixed

2013-10-21 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #2421 Status: Fixed Duration: 1751 seconds Commit: 1ead64469a8942d11d56eb2c0abb9cfc4aa60508 (trunk) Author: Thomas Mueller Message: OAK-825 Unnecessary performance overhead (too many child nodes are traversed) g

jackrabbit-oak build #2422: Fixed

2013-10-21 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #2422 Status: Fixed Duration: 1922 seconds Commit: ad80c977f15843097b53148952a8c6206a6fd47f (trunk) Author: Thomas Mueller Message: OAK-641: Improved benchmark tooling - save before running the test, use a non-or

jackrabbit-oak build #2423: Fixed

2013-10-21 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #2423 Status: Fixed Duration: 2088 seconds Commit: 3497dad7ffe8b11ae8693ff9707dad91453af53b (trunk) Author: Thomas Mueller Message: OAK-641: Improved benchmark tooling - use different node names git-svn-id: https

jackrabbit-oak build #2424: Broken

2013-10-21 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #2424 Status: Broken Duration: 1091 seconds Commit: 7bca7b45a1b7e6521b9cc3a4c422cf5848f2833b (trunk) Author: Angela Schreiber Message: OAK-527: permissions (minor improvement) git-svn-id: https://svn.apache.org/r

Re: jackrabbit-oak build #2424: Broken

2013-10-21 Thread Angela Schreiber
i will look at it On 10/21/13 5:26 PM, "Travis CI" wrote: >Build Update for apache/jackrabbit-oak >- > >Build: #2424 >Status: Broken > >Duration: 1091 seconds >Commit: 7bca7b45a1b7e6521b9cc3a4c422cf5848f2833b (trunk) >Author: Angela Schreiber >Message: OAK-527

Re: Oak JCR Observation scalability aspects and concerns

2013-10-21 Thread Carsten Ziegeler
This Sling listener is providing higher level application support as it creates resource events out of jcr observation events. A lot of code in Sling now relies on this functionality and even more higher level code based on Sling does so. Usually application listeners register for "/" and then filt

Re: jackrabbit-oak build #2424: Broken

2013-10-21 Thread Angela Schreiber
hmm... the build works for me including the integrationtests. regards angela On 10/21/13 5:52 PM, "Angela Schreiber" wrote: >i will look at it > >On 10/21/13 5:26 PM, "Travis CI" wrote: > >>Build Update for apache/jackrabbit-oak >>- >> >>Build: #2424 >>Statu

Re: Oak JCR Observation scalability aspects and concerns

2013-10-21 Thread Jukka Zitting
Hi, On Mon, Oct 21, 2013 at 12:03 PM, Carsten Ziegeler wrote: > This Sling listener is providing higher level application support as it > creates resource events out of jcr observation events. A lot of code in > Sling now relies on this functionality and even more higher level code > based on Sli

Re: Detecting move operations in node state diffs

2013-10-21 Thread Michael Dürig
On 21.10.13 7:18 , Tobias Bocanegra wrote: On Mon, Oct 21, 2013 at 7:31 AM, Thomas Mueller wrote: >Hi, > >>extra pass > >On how to avoid this extra pass. Not strictly backward compatible, but I >wonder how much it would break: what if observation would deliver two >events for moved nodes: t

Re: Oak JCR Observation scalability aspects and concerns

2013-10-21 Thread Angela Schreiber
hi just one more comment regarding caches from a security perspective >> However, there are many use cases where a local cache is used on >> each instance which would make such an approach useless. > >Oak has a variety of solutions that could be used to make such code >more scalable (ordered by i