Hi Richard- I have a embedded jetty OSGi component that exposes the conglomeration of OSGi services to the outside world. Felix would need to set these service references in a thread-safe way. You appear to be suggesting that Felix can be only consumed by the Felix threads themselves, as only that would not bypass 'the hidden synchronization inside the framework' - whatever that is.
Let me try one more time: a while back, Felix mandated that DYNAMIC @References be volatile. Presumably that decision was made by someone who knows the JMM, and thus could provide a response to the question in my initial post: Perhaps an explanation as to why a DYNAMIC @Reference does need to be volatile would be helpful - e.g. why does component de-activation/re-activation provide memory visibility guarantees for STATIC references? I don't mean to be contentious, and there is nothing preventing me from declaring my @References volatile. I just was hoping that there was JMM-knowledge active in this forum which could provide an answer to my original question. I may well have my answer. Thanks Dirk On Tue, May 2, 2017 at 2:29 PM, Richard S. Hall <he...@ungoverned.org> wrote: > Dirk, > > Allow I'm not well versed on the details of JMM. I think this boils down > to typical OSGi component code will be going through the hidden barriers in > the the framework when they try to retrieve and access the service objects. > You appear to be concocting a situation where you have threads from outside > that you have given access to OSGi service components and bypassing the > hidden synchronization inside the framework. If you can create such a > scenario, then it is up to you to safely publish these objects when you > hand them off to this other thread. And the only way this is really > problematic is if your handoff is done in such a way that you bypass all > barriers to begin with. > > -> richard > > > > On 5/2/17 16:27 , Dirk Hogan wrote: > >> Hi Neil- >> OK - now we are getting somewhere. >> >> You need to dig a bit deeper to see what establishes a happens-before >> relationship. For within a single thread, it is the Program Order: the >> underlying virtual machine cannot allow the visibility of memory effects >> which contradict the sequence of instructions defined in your .java file, >> even though these commands will be re-ordered by the compiler, and >> pipelined and executed concurrently by machine cores. >> >> Between threads, it is the Synchronization Order, which is NOT Program >> Order. Between threads, the memory effects of your .java file writes/reads >> do NOT have to be honored. So if you say 'But Felix writes the reference >> before publishing the service', and thus concluding that the reference >> will >> always be present in the service, you are correct, but ONLY from the >> perspective of the thread which wrote the reference and published the >> service. For ALL other threads, all bets are off, unless there is a >> Synchronization Action (volatile reference read/write, monitor lock/unlock >> on the same monitor) which dictates a Synchronization Order of the >> visibility of memory effects between DISTINCT threads. >> >> The definition of Synchronization Actions, how they combine to create a >> Synchronization Order, and how this, in turn, allows a happens-before >> relationship to be established, is a formal set of rules defined in the >> JMM. There is no ambiguity here. A read of section 17.4 will illuminate as >> much. It is very dense - that is why I turned to the other references I >> provided in the links above. They are not a quick/easy read, but are >> necessary to coherently reason about memory visibility across threads. >> >> Thanks >> >> Dirk >> >> On Tue, May 2, 2017 at 1:10 PM, Neil Bartlett <njbartl...@gmail.com> >> wrote: >> >> I think I get what Dirk is saying, and although I disagree with some of >>> his analysis there is arguably a gap in the DS specification that makes >>> this unclear. I hope somebody can point at the vital sentence I have >>> missed, but here is my analysis: >>> >>> The JLS defines happens-before thus: "Two actions can be ordered by a >>> happens-before relationship. If one action happens-before another, then >>> the >>> first is visible to and ordered before the second.” ( >>> https://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jls-17.4.5 >>> < >>> https://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jls-17.4.5 >>> >>>> ). >>>> >>> Define two actions, A1 and A2. A1 is the setting of an @Reference field >>> in >>> a component. A2 is reading the value of that field. It is Dirk’s >>> contention >>> that no provable happens-before relationship edge exists between A1 and >>> A2, >>> and therefore it’s possible for A2 to access the value of the @Reference >>> field that obtained before A1. Let’s see. >>> >>> A1 is an action that is taken by SCR on an object that it creates (the >>> component instance). A2 can only occur in the following circumstances: >>> >>> 1. SCR itself reads the field. >>> SCR is in full control of its own code and I am assuming a correct SCR >>> implementation. >>> >>> 2. The activate method of the component uses the value of the field. >>> Section 112.3.8.1 of OSGi Compendium R6 states "there is a happens-before >>> relationship between setting the field and activating the component >>> instance”. Also section 112.3.6.1 states “The bind method is called >>> and/or >>> the field is set before the component instance is activated”, which is >>> less >>> clear about the happens-before but I think the intent is there, and it >>> also >>> clarifies that this applies to bind methods as well as injected fields. >>> Therefore action A1 happens-before A2 in this case. >>> >>> Note that the spec does not tell the implementation HOW to achieve this >>> happens-before edge. As Dirk points out, if it is done with a >>> synchronized >>> block then it would have to be on the same monitor. For example, the >>> implementation can achieve this by synchronizing on the component >>> instance >>> itself when binding fields and calling the activate method. >>> >>> 3. The activate method starts a thread which uses the value of the field >>> (I think this maps to Dirk’s HTTP-serving example). >>> From point 2 above, we know that A1 happens-before the activate method >>> is >>> invoked. JLS 17.4.5 states that “a call to start() on a thread >>> happens-before any actions in the started thread”. JLS also states that >>> happens-before is transitive, i.e. if X happens-before Y and Y >>> happens-before Z then X happens-before Z. >>> >>> Therefore A1 happens-before A2 in this case. >>> >>> 4. Service method(s) on the component use the value of the field. >>> The analysis here diverges for immediate vs delayed components. >>> >>> For immediate components, Compendium section 112.5.3 states: "If an >>> immediate component configuration is satisfied and specifies a service, >>> SCR >>> must register the component configuration as a service in the service >>> registry and then activate the component configuration”. This does NOT >>> state that the registration of the service must happen after the binding >>> of >>> the reference fields. So it seems an implementation is free to hand the >>> service object to a consumer before the static references are bound. This >>> surprised me. >>> >>> For delayed components, Compendium section 112.5.4 states: "When the >>> service is requested … SCR must create and activate a [unique] component >>> configuration”. It does NOT explicitly state that the static references >>> must be bound before the service object is handed to the caller of >>> getService(). Again this surprised me. >>> >>> >>> In summary, it’s clear to me that the *intention* of the specification is >>> for static references to be bound before the service becomes accessible, >>> and that component writers should not have to declare those fields >>> volatile, but this intention is not adequately spelled out. The best >>> statement I could find is in section 112.3.6.1: "A component instance >>> never sees any of the dynamics of the static reference”, which is very >>> vague. >>> >>> Again, I may well have missed something or made an error in my analysis. >>> I >>> look forward to be being corrected. >>> >>> >>> Neil >>> >>> >>> On 2 May 2017, at 19:56, David Jencks <david.a.jen...@gmail.com> wrote: >>>> >>>> I don’t understand your example or I don’t understand how your example >>>> >>> relates to an osgi or ds problem. >>> >>>> I think you are proposing that the DS component is an http server and >>>> >>> starts threads etc. If so, it’s that components responsibility to make >>> sure its’ internal operations are thread safe, and this has little to do >>> with osgi or ds. >>> >>>> If you are asking about osgi, for instance if the component exposes >>>> >>> HttpServlet and is registered with the whiteboard http service and >>> receives >>> calls from it, then the whiteboard had to call getService to get the DS >>> component instance. In this case I think that there are plenty of >>> synchronization barriers in the service registry and in e.g. >>> SingleComponentManager.getService to assure that the view a request >>> thread has of the component is the same as the view the activation thread >>> had. >>> >>>> I could be wrong about whether there are actually enough synchronization >>>> >>> barriers. Have you looked at the code to see? >>> >>>> thanks >>>> david jencks >>>> >>>> On May 2, 2017, at 10:45 AM, Dirk Hogan <dirk.ho...@forgerock.com> >>>>> >>>> wrote: >>> >>>> Hi Raymond- >>>>> Assume Felix does it's thing: sets all @References, activates all >>>>> components, publishes all services. It has resolved all dependencies, >>>>> >>>> and >>> >>>> set references accordingly. >>>>> >>>>> Thus, in the example above, myOSGiComponent has its @Reference to >>>>> SomeOtherOSGiService resolved - Felix has set this reference. At the >>>>> conclusion of the activate method, myOSGiComponent has exposed some >>>>> functionality via http, even successfully dereferenced myOSGiComponent, >>>>> >>>> and >>> >>>> now requests start to come it, serviced by the http-server's >>>>> >>>> thread-pool. >>> >>>> Part of servicing this request involves dereferencing (reading) the >>>>> @Reference myServiceReference, which, I agree has been set (written) by >>>>> >>>> a >>> >>>> Felix thread. >>>>> >>>>> The point of the Java Memory Model is that memory state, including >>>>> references, read and written across threads, do not have to have >>>>> determinate state, unless there is a Synchronization Action which >>>>> >>>> ensures >>> >>>> that the state written by thread A is visible to any other thread - a >>>>> volatile reference provides such a Synchronization Action. So code in >>>>> myOSGiComponent activate method can successfully dereference the >>>>> SomeOtherOSGiService reference, because the same thread which set the >>>>> reference is calling activate. But this does NOT mean that another >>>>> >>>> thread, >>> >>>> reading the SAME reference, will see a properly initialized reference. >>>>> >>>>> Again, this should be self-evident, and the various rules of the JMM >>>>> >>>> model >>> >>>> are defined in section 17.4 of the Java Language Specification. If it is >>>>> not clear, then an understanding if the JMM is required. This >>>>> >>>> understanding >>> >>>> can be gained by reading the JLS, reading the links above, the chapter >>>>> in *Concurrency >>>>> in Practice* on the JMM, or http://gee.cs.oswego.edu/dl/cpj/jmm.html >>>>> provides a good intro. >>>>> >>>>> The bottom line: memory state, including references, shared among >>>>> >>>> multiple >>> >>>> threads can have indeterminate state when written/read, unless explicit >>>>> steps, defined in the JMM, are taken to ensure this visibility and thus >>>>> coherent state. This is not an opinion. >>>>> >>>>> Thanks >>>>> >>>>> Dirk >>>>> >>>>> >>>>> >>>>> On Tue, May 2, 2017 at 10:23 AM, Raymond Auge < >>>>> raymond.a...@liferay.com >>>>> wrote: >>>>> >>>>> On Tue, May 2, 2017 at 1:18 PM, Raymond Auge <raymond.a...@liferay.com >>>>>> wrote: >>>>>> >>>>>> >>>>>>> On Tue, May 2, 2017 at 1:14 PM, Raymond Auge < >>>>>>> >>>>>> raymond.a...@liferay.com> >>> >>>> wrote: >>>>>>> >>>>>>> On Tue, May 2, 2017 at 11:46 AM, Dirk Hogan < >>>>>>>> >>>>>>> dirk.ho...@forgerock.com> >>> >>>> wrote: >>>>>>>> >>>>>>>> But no similar guarantee applies when another thread hits the >>>>>>>>> component, >>>>>>>>> >>>>>>>>> I believe the discussion boils down to why you believe the above is >>>>>>>> possible? >>>>>>>> >>>>>>>> If SCR has not yet made the component available to other threads >>>>>>>> >>>>>>> (which, >>> >>>> to my knowledge, it only does by registering it as a service to the >>>>>>>> >>>>>>> service >>>>>> >>>>>>> registry) how can _another_ thread access the component at all? >>>>>>>> >>>>>>>> I don't even see a way for components within the same DS bundle to >>>>>>>> interact with each other other than via the service registry because >>>>>>>> @Reference only works through the registry. >>>>>>>> >>>>>>>> Correction, components could interact with each other via the >>>>>>> ComponentContext.. but I guess we'd have to know if that's part of >>>>>>> >>>>>> your >>> >>>> use >>>>>> >>>>>>> case. >>>>>>> >>>>>>> Actually, no, it's not even possible via the ComponentContext. It >>>>>> only >>>>>> allows access to Services. >>>>>> >>>>>> - Ray >>>>>>> >>>>>>> >>>>>>> Please, can you explain in more detail what case might allow for the >>>>>>>> above? Because I believe the assumption is that it's not currently >>>>>>>> possible. Certainly there could be a bug and I'm certain everyone >>>>>>>> >>>>>>> would >>> >>>> want to see that fixed. >>>>>>>> >>>>>>>> -- >>>>>>>> *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile> >>>>>>>> (@rotty3000) >>>>>>>> Senior Software Architect *Liferay, Inc.* <http://www.liferay.com> >>>>>>>> (@Liferay) >>>>>>>> Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org> >>>>>>>> (@OSGiAlliance) >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile> >>>>>>> (@rotty3000) >>>>>>> Senior Software Architect *Liferay, Inc.* <http://www.liferay.com> >>>>>>> (@Liferay) >>>>>>> Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org> >>>>>>> (@OSGiAlliance) >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile> >>>>>> (@rotty3000) >>>>>> Senior Software Architect *Liferay, Inc.* <http://www.liferay.com> >>>>>> (@Liferay) >>>>>> Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org> >>>>>> (@OSGiAlliance) >>>>>> >>>>>> >>> >
