Observation: External vs local - Load distribution

2015-06-15 Thread Carsten Ziegeler
Hi, with JR2 there was a clear distinction between local and external events. A pattern that emerged out of this is to use the "local" information of an observation event to start some processing. By using this information you have a load distribution of the processing which is equal to the distri

Re: Observation: External vs local - Load distribution

2015-06-15 Thread Marcel Reutegger
Hi, you can write a CommitEditor, which is called with every local commit. Regards Marcel On 15/06/15 09:43, "Carsten Ziegeler" wrote: >Hi, > >with JR2 there was a clear distinction between local and external >events. A pattern that emerged out of this is to use the "local" >information of an

Re: Observation: External vs local - Load distribution

2015-06-15 Thread Carsten Ziegeler
Am 15.06.15 um 14:23 schrieb Marcel Reutegger: > Hi, > > you can write a CommitEditor, which is called with every > local commit. > Is it easy to calculate the changed nodes/properties in this editor? Carsten > Regards > Marcel > > On 15/06/15 09:43, "Carsten Ziegeler" wrote: > >> Hi, >> >>

Re: Observation: External vs local - Load distribution

2015-06-15 Thread Stefan Egli
On 6/15/15 2:40 PM, "Carsten Ziegeler" wrote: >Am 15.06.15 um 14:23 schrieb Marcel Reutegger: >> Hi, >> >> you can write a CommitEditor, which is called with every >> local commit. >> > >Is it easy to calculate the changed nodes/properties in this editor? As I understand yes, the Editor gets c

Re: Observation: External vs local - Load distribution

2015-06-15 Thread Carsten Ziegeler
Am 15.06.15 um 14:50 schrieb Stefan Egli: > On 6/15/15 2:40 PM, "Carsten Ziegeler" wrote: > >> Am 15.06.15 um 14:23 schrieb Marcel Reutegger: >>> Hi, >>> >>> you can write a CommitEditor, which is called with every >>> local commit. >>> >> >> Is it easy to calculate the changed nodes/properties i

Re: Observation: External vs local - Load distribution

2015-06-15 Thread Julian Sedding
AFAIK the CommitEditor gets called before the commit is finalized/persisted. Later CommitEditors can still fail the commit. This may lead to events being reported, that did not actually happen. It needs to be considered if this represents a problem for the given use-case. Regards Julian On Mon,

Re: Observation: External vs local - Load distribution

2015-06-15 Thread Carsten Ziegeler
Am 15.06.15 um 15:23 schrieb Julian Sedding: > AFAIK the CommitEditor gets called before the commit is > finalized/persisted. Later CommitEditors can still fail the commit. > This may lead to events being reported, that did not actually happen. > > It needs to be considered if this represents a pr

Re: Observation: External vs local - Load distribution

2015-06-15 Thread Chetan Mehrotra
On Mon, Jun 15, 2015 at 1:13 PM, Carsten Ziegeler wrote: > Now, with Oak there is still this distinction, however if I remember > correctly under heavy load it might happen that local events are > reported as external events. And in that case the above pattern fails. > Regardless of how rare this

Re: Observation: External vs local - Load distribution

2015-06-15 Thread Carsten Ziegeler
Am 15.06.15 um 16:21 schrieb Chetan Mehrotra: > On Mon, Jun 15, 2015 at 1:13 PM, Carsten Ziegeler > wrote: >> Now, with Oak there is still this distinction, however if I remember >> correctly under heavy load it might happen that local events are >> reported as external events. And in that case t

Re: Observation: External vs local - Load distribution

2015-06-15 Thread Stefan Egli
On 6/15/15 4:29 PM, "Carsten Ziegeler" wrote: >Am 15.06.15 um 16:21 schrieb Chetan Mehrotra: >> On Mon, Jun 15, 2015 at 1:13 PM, Carsten Ziegeler >> wrote: >>> Now, with Oak there is still this distinction, however if I remember >>> correctly under heavy load it might happen that local events are

Re: Observation: External vs local - Load distribution

2015-06-15 Thread Carsten Ziegeler
Am 15.06.15 um 17:31 schrieb Stefan Egli: > >>> >> Ah, thanks Chetan, that's the first time I hear this - so basically if >> we implement our own observer, we can reliably get: >> a) all changes >> b) local/external info >> c) user id >> >> Is that correct? > > the way I understand it is: ;) >

Re: Observation: External vs local - Load distribution

2015-06-17 Thread Carsten Ziegeler
Ok, just to recap. In Sling we can implement the Observer interface (and not use the BackgroundObserver base class). This will give us reliably user id for all local events. Does anyone see a problem with this approach? Carsten -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org

Re: Observation: External vs local - Load distribution

2015-06-17 Thread Chetan Mehrotra
Just ensure that your Observer is fast as its invoked the critical path. This would probably end up with a design similar to Background Observer. May be better option would be to allow BO have non bounded queue. Chetan Mehrotra On Wed, Jun 17, 2015 at 2:05 PM, Carsten Ziegeler wrote: > Ok, just

Re: Observation: External vs local - Load distribution

2015-06-17 Thread Ian Boston
Hi, +1 for unbounded, let the GC take care of it and log periodically when its size becomes significant so that anyone wondering why their JVM is consuming so much GC time gets a clue as to the cause, without having to perform heap dumps, thread dumps or jvm probes. (but ideally all queues would

Re: Observation: External vs local - Load distribution

2015-06-29 Thread Michael Dürig
On 17.6.15 7:47 , Ian Boston wrote: Hi, +1 for unbounded, let the GC take care of it and log periodically when its size becomes significant so that anyone wondering why their JVM is consuming so much GC time gets a clue as to the cause, without having to perform heap dumps, thread dumps or jvm

Re: Observation: External vs local - Load distribution

2015-06-29 Thread Michael Dürig
On 15.6.15 5:46 , Carsten Ziegeler wrote: And it would be great if that implementation could simply pass all filters down to the oak level +1. Oak provides a rich and extensible interface for filtering observation events [1]. org.apache.jackrabbit.oak.plugins.observation.filter.FilterBuilde

Re: Observation: External vs local - Load distribution

2015-10-13 Thread Carsten Ziegeler
Am 17.06.15 um 10:35 schrieb Carsten Ziegeler: > Ok, just to recap. In Sling we can implement the Observer interface (and > not use the BackgroundObserver base class). This will give us reliably > user id for all local events. > > Does anyone see a problem with this approach? > Getting back to th

Re: Observation: External vs local - Load distribution

2015-10-13 Thread Stefan Egli
Hi Carsten, For external events the commit info is indeed not provided yup. For internal ones it is - except for those 'overflow' ones which collapse into a pseudo-external one. Cheers, Stefan On 13/10/15 15:17, "Carsten Ziegeler" wrote: >Am 17.06.15 um 10:35 schrieb Carsten Ziegeler: >> Ok, j