I have no stomach for further discussion/argument.
Thanks for the responses guys.
Dirk
On Tue, May 2, 2017 at 4:50 PM, Richard S. Hall <he...@ungoverned.org>
wrote:
> On 5/2/17 19:41 , Richard S. Hall wrote:
>
>> Resending to the mailing list...
>>
>> On 5/2/17 18:55 , Dirk Hogan wrote:
>>
>>> Hi Richard-
>>> I have a embedded jetty OSGi component that exposes the conglomeration of
>>> OSGi services to the outside world. Felix would need to set these service
>>> references in a thread-safe way. You appear to be suggesting that Felix
>>> can
>>> be only consumed by the Felix threads themselves, as only that would not
>>> bypass 'the hidden synchronization inside the framework' - whatever that
>>> is.
>>>
>>
>> I'm suggesting that if you pass objects to threads then you need to do so
>> in a thread safe way.
>>
>> Assume some something like:
>>
>> class FooThread {
>> private MyService service;
>> pubic void run() {
>> while (true) {
>> if (service != null) {
>> service.sayHello();
>> }
>> }
>> }
>> public void setService(MyService s) {
>> service = s;
>> }
>> }
>>
>> So lets assume that there is some thread spinning on run(). If in another
>> component you create a FooThread then get a service object from the service
>> registry and call myFooThread.setService() (where the MyService instance
>> contains your injected service), who is responsible for safely publishing
>> your service to your FooThread?
>>
>> Clearly, if you just call setService() you are not safely handing off the
>> service object to your thread (this is the equivalent of your shutdown flag
>> example). You are implying that it is the frameworks responsibility to
>> provide a barrier to FooThreads member service variable.
>>
>> Certainly, one solution to this is making your reference volatile, but
>> that it between you and FooThread, not between your service and the
>> framework.
>>
>
> Just some further elaboration and generalization, since the above may be
> confusing...
>
> The main point is you have some thread that you somehow have given access
> to an object and internally that object invokes some OSGi service. If you
> hand that object reference to your thread in a safe way, then there will be
> no problem.
>
> At the end of the day, the thread invoking this object had to be given the
> object reference by you, so you are responsible for giving it the reference
> using a barrier.
>
> -> richard
>
>
>
>> -> richard
>>
>>
>>> Let me try one more time: a while back, Felix mandated that DYNAMIC
>>> @References be volatile. Presumably that decision was made by someone who
>>> knows the JMM, and thus could provide a response to the question in my
>>> initial post:
>>>
>>> Perhaps an explanation as to why a DYNAMIC @Reference does need to be
>>> volatile would be helpful - e.g. why does component
>>> de-activation/re-activation provide memory visibility guarantees for
>>> STATIC
>>> references?
>>>
>>> I don't mean to be contentious, and there is nothing preventing me from
>>> declaring my @References volatile. I just was hoping that there was
>>> JMM-knowledge active in this forum which could provide an answer to my
>>> original question. I may well have my answer.
>>> Thanks
>>>
>>> Dirk
>>>
>>>
>>> On Tue, May 2, 2017 at 2:29 PM, Richard S. Hall <he...@ungoverned.org>
>>> wrote:
>>>
>>> Dirk,
>>>>
>>>> Allow I'm not well versed on the details of JMM. I think this boils down
>>>> to typical OSGi component code will be going through the hidden
>>>> barriers in
>>>> the the framework when they try to retrieve and access the service
>>>> objects.
>>>> You appear to be concocting a situation where you have threads from
>>>> outside
>>>> that you have given access to OSGi service components and bypassing the
>>>> hidden synchronization inside the framework. If you can create such a
>>>> scenario, then it is up to you to safely publish these objects when you
>>>> hand them off to this other thread. And the only way this is really
>>>> problematic is if your handoff is done in such a way that you bypass all
>>>> barriers to begin with.
>>>>
>>>> -> richard
>>>>
>>>>
>>>>
>>>> On 5/2/17 16:27 , Dirk Hogan wrote:
>>>>
>>>> Hi Neil-
>>>>> OK - now we are getting somewhere.
>>>>>
>>>>> You need to dig a bit deeper to see what establishes a happens-before
>>>>> relationship. For within a single thread, it is the Program Order: the
>>>>> underlying virtual machine cannot allow the visibility of memory
>>>>> effects
>>>>> which contradict the sequence of instructions defined in your .java
>>>>> file,
>>>>> even though these commands will be re-ordered by the compiler, and
>>>>> pipelined and executed concurrently by machine cores.
>>>>>
>>>>> Between threads, it is the Synchronization Order, which is NOT Program
>>>>> Order. Between threads, the memory effects of your .java file
>>>>> writes/reads
>>>>> do NOT have to be honored. So if you say 'But Felix writes the
>>>>> reference
>>>>> before publishing the service', and thus concluding that the reference
>>>>> will
>>>>> always be present in the service, you are correct, but ONLY from the
>>>>> perspective of the thread which wrote the reference and published the
>>>>> service. For ALL other threads, all bets are off, unless there is a
>>>>> Synchronization Action (volatile reference read/write, monitor
>>>>> lock/unlock
>>>>> on the same monitor) which dictates a Synchronization Order of the
>>>>> visibility of memory effects between DISTINCT threads.
>>>>>
>>>>> The definition of Synchronization Actions, how they combine to create a
>>>>> Synchronization Order, and how this, in turn, allows a happens-before
>>>>> relationship to be established, is a formal set of rules defined in the
>>>>> JMM. There is no ambiguity here. A read of section 17.4 will
>>>>> illuminate as
>>>>> much. It is very dense - that is why I turned to the other references I
>>>>> provided in the links above. They are not a quick/easy read, but are
>>>>> necessary to coherently reason about memory visibility across threads.
>>>>>
>>>>> Thanks
>>>>>
>>>>> Dirk
>>>>>
>>>>> On Tue, May 2, 2017 at 1:10 PM, Neil Bartlett <njbartl...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> I think I get what Dirk is saying, and although I disagree with some of
>>>>>
>>>>>> his analysis there is arguably a gap in the DS specification that
>>>>>> makes
>>>>>> this unclear. I hope somebody can point at the vital sentence I have
>>>>>> missed, but here is my analysis:
>>>>>>
>>>>>> The JLS defines happens-before thus: "Two actions can be ordered by a
>>>>>> happens-before relationship. If one action happens-before another,
>>>>>> then
>>>>>> the
>>>>>> first is visible to and ordered before the second.” (
>>>>>> https://docs.oracle.com/javase/specs/jls/se7/html/jls-17.
>>>>>> html#jls-17.4.5
>>>>>> <
>>>>>> https://docs.oracle.com/javase/specs/jls/se7/html/jls-17.
>>>>>> html#jls-17.4.5
>>>>>>
>>>>>> ).
>>>>>>>
>>>>>>> Define two actions, A1 and A2. A1 is the setting of an @Reference
>>>>>> field
>>>>>> in
>>>>>> a component. A2 is reading the value of that field. It is Dirk’s
>>>>>> contention
>>>>>> that no provable happens-before relationship edge exists between A1
>>>>>> and
>>>>>> A2,
>>>>>> and therefore it’s possible for A2 to access the value of the
>>>>>> @Reference
>>>>>> field that obtained before A1. Let’s see.
>>>>>>
>>>>>> A1 is an action that is taken by SCR on an object that it creates (the
>>>>>> component instance). A2 can only occur in the following circumstances:
>>>>>>
>>>>>> 1. SCR itself reads the field.
>>>>>> SCR is in full control of its own code and I am assuming a correct SCR
>>>>>> implementation.
>>>>>>
>>>>>> 2. The activate method of the component uses the value of the field.
>>>>>> Section 112.3.8.1 of OSGi Compendium R6 states "there is a
>>>>>> happens-before
>>>>>> relationship between setting the field and activating the component
>>>>>> instance”. Also section 112.3.6.1 states “The bind method is called
>>>>>> and/or
>>>>>> the field is set before the component instance is activated”, which is
>>>>>> less
>>>>>> clear about the happens-before but I think the intent is there, and it
>>>>>> also
>>>>>> clarifies that this applies to bind methods as well as injected
>>>>>> fields.
>>>>>> Therefore action A1 happens-before A2 in this case.
>>>>>>
>>>>>> Note that the spec does not tell the implementation HOW to achieve
>>>>>> this
>>>>>> happens-before edge. As Dirk points out, if it is done with a
>>>>>> synchronized
>>>>>> block then it would have to be on the same monitor. For example, the
>>>>>> implementation can achieve this by synchronizing on the component
>>>>>> instance
>>>>>> itself when binding fields and calling the activate method.
>>>>>>
>>>>>> 3. The activate method starts a thread which uses the value of the
>>>>>> field
>>>>>> (I think this maps to Dirk’s HTTP-serving example).
>>>>>> From point 2 above, we know that A1 happens-before the activate
>>>>>> method
>>>>>> is
>>>>>> invoked. JLS 17.4.5 states that “a call to start() on a thread
>>>>>> happens-before any actions in the started thread”. JLS also states
>>>>>> that
>>>>>> happens-before is transitive, i.e. if X happens-before Y and Y
>>>>>> happens-before Z then X happens-before Z.
>>>>>>
>>>>>> Therefore A1 happens-before A2 in this case.
>>>>>>
>>>>>> 4. Service method(s) on the component use the value of the field.
>>>>>> The analysis here diverges for immediate vs delayed components.
>>>>>>
>>>>>> For immediate components, Compendium section 112.5.3 states: "If an
>>>>>> immediate component configuration is satisfied and specifies a
>>>>>> service,
>>>>>> SCR
>>>>>> must register the component configuration as a service in the service
>>>>>> registry and then activate the component configuration”. This does NOT
>>>>>> state that the registration of the service must happen after the
>>>>>> binding
>>>>>> of
>>>>>> the reference fields. So it seems an implementation is free to hand
>>>>>> the
>>>>>> service object to a consumer before the static references are bound.
>>>>>> This
>>>>>> surprised me.
>>>>>>
>>>>>> For delayed components, Compendium section 112.5.4 states: "When the
>>>>>> service is requested … SCR must create and activate a [unique]
>>>>>> component
>>>>>> configuration”. It does NOT explicitly state that the static
>>>>>> references
>>>>>> must be bound before the service object is handed to the caller of
>>>>>> getService(). Again this surprised me.
>>>>>>
>>>>>>
>>>>>> In summary, it’s clear to me that the *intention* of the
>>>>>> specification is
>>>>>> for static references to be bound before the service becomes
>>>>>> accessible,
>>>>>> and that component writers should not have to declare those fields
>>>>>> volatile, but this intention is not adequately spelled out. The best
>>>>>> statement I could find is in section 112.3.6.1: "A component instance
>>>>>> never sees any of the dynamics of the static reference”, which is very
>>>>>> vague.
>>>>>>
>>>>>> Again, I may well have missed something or made an error in my
>>>>>> analysis.
>>>>>> I
>>>>>> look forward to be being corrected.
>>>>>>
>>>>>>
>>>>>> Neil
>>>>>>
>>>>>>
>>>>>> On 2 May 2017, at 19:56, David Jencks <david.a.jen...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I don’t understand your example or I don’t understand how your
>>>>>>> example
>>>>>>>
>>>>>>> relates to an osgi or ds problem.
>>>>>>
>>>>>> I think you are proposing that the DS component is an http server and
>>>>>>>
>>>>>>> starts threads etc. If so, it’s that components responsibility to
>>>>>> make
>>>>>> sure its’ internal operations are thread safe, and this has little to
>>>>>> do
>>>>>> with osgi or ds.
>>>>>>
>>>>>> If you are asking about osgi, for instance if the component exposes
>>>>>>>
>>>>>>> HttpServlet and is registered with the whiteboard http service and
>>>>>> receives
>>>>>> calls from it, then the whiteboard had to call getService to get the
>>>>>> DS
>>>>>> component instance. In this case I think that there are plenty of
>>>>>> synchronization barriers in the service registry and in e.g.
>>>>>> SingleComponentManager.getService to assure that the view a request
>>>>>> thread has of the component is the same as the view the activation
>>>>>> thread
>>>>>> had.
>>>>>>
>>>>>> I could be wrong about whether there are actually enough
>>>>>>> synchronization
>>>>>>>
>>>>>>> barriers. Have you looked at the code to see?
>>>>>>
>>>>>> thanks
>>>>>>> david jencks
>>>>>>>
>>>>>>> On May 2, 2017, at 10:45 AM, Dirk Hogan <dirk.ho...@forgerock.com>
>>>>>>> wrote:
>>>>>>> Hi Raymond-
>>>>>>>
>>>>>>>> Assume Felix does it's thing: sets all @References, activates all
>>>>>>>> components, publishes all services. It has resolved all
>>>>>>>> dependencies,
>>>>>>>>
>>>>>>>> and
>>>>>>> set references accordingly.
>>>>>>>
>>>>>>>> Thus, in the example above, myOSGiComponent has its @Reference to
>>>>>>>> SomeOtherOSGiService resolved - Felix has set this reference. At the
>>>>>>>> conclusion of the activate method, myOSGiComponent has exposed some
>>>>>>>> functionality via http, even successfully dereferenced
>>>>>>>> myOSGiComponent,
>>>>>>>>
>>>>>>>> and
>>>>>>> now requests start to come it, serviced by the http-server's
>>>>>>> thread-pool.
>>>>>>> Part of servicing this request involves dereferencing (reading) the
>>>>>>>
>>>>>>>> @Reference myServiceReference, which, I agree has been set
>>>>>>>> (written) by
>>>>>>>>
>>>>>>>> a
>>>>>>> Felix thread.
>>>>>>>
>>>>>>>> The point of the Java Memory Model is that memory state, including
>>>>>>>> references, read and written across threads, do not have to have
>>>>>>>> determinate state, unless there is a Synchronization Action which
>>>>>>>>
>>>>>>>> ensures
>>>>>>> that the state written by thread A is visible to any other thread - a
>>>>>>>
>>>>>>>> volatile reference provides such a Synchronization Action. So code
>>>>>>>> in
>>>>>>>> myOSGiComponent activate method can successfully dereference the
>>>>>>>> SomeOtherOSGiService reference, because the same thread which set
>>>>>>>> the
>>>>>>>> reference is calling activate. But this does NOT mean that another
>>>>>>>>
>>>>>>>> thread,
>>>>>>> reading the SAME reference, will see a properly initialized
>>>>>>> reference.
>>>>>>>
>>>>>>>> Again, this should be self-evident, and the various rules of the JMM
>>>>>>>>
>>>>>>>> model
>>>>>>> are defined in section 17.4 of the Java Language Specification. If
>>>>>>> it is
>>>>>>>
>>>>>>>> not clear, then an understanding if the JMM is required. This
>>>>>>>>
>>>>>>>> understanding
>>>>>>> can be gained by reading the JLS, reading the links above, the
>>>>>>> chapter
>>>>>>>
>>>>>>>> in *Concurrency
>>>>>>>> in Practice* on the JMM, or http://gee.cs.oswego.edu/dl/cp
>>>>>>>> j/jmm.html
>>>>>>>> provides a good intro.
>>>>>>>>
>>>>>>>> The bottom line: memory state, including references, shared among
>>>>>>>>
>>>>>>>> multiple
>>>>>>> threads can have indeterminate state when written/read, unless
>>>>>>> explicit
>>>>>>>
>>>>>>>> steps, defined in the JMM, are taken to ensure this visibility and
>>>>>>>> thus
>>>>>>>> coherent state. This is not an opinion.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> Dirk
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, May 2, 2017 at 10:23 AM, Raymond Auge <
>>>>>>>> raymond.a...@liferay.com
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On Tue, May 2, 2017 at 1:18 PM, Raymond Auge <
>>>>>>>> raymond.a...@liferay.com
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, May 2, 2017 at 1:14 PM, Raymond Auge <
>>>>>>>>>>
>>>>>>>>>> raymond.a...@liferay.com>
>>>>>>>>>
>>>>>>>> wrote:
>>>>>>>
>>>>>>>> On Tue, May 2, 2017 at 11:46 AM, Dirk Hogan <
>>>>>>>>>> dirk.ho...@forgerock.com>
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>
>>>>>>>> But no similar guarantee applies when another thread hits the
>>>>>>>>>>>
>>>>>>>>>>>> component,
>>>>>>>>>>>>
>>>>>>>>>>>> I believe the discussion boils down to why you believe the
>>>>>>>>>>>> above is
>>>>>>>>>>>>
>>>>>>>>>>> possible?
>>>>>>>>>>>
>>>>>>>>>>> If SCR has not yet made the component available to other threads
>>>>>>>>>>>
>>>>>>>>>>> (which,
>>>>>>>>>>
>>>>>>>>> to my knowledge, it only does by registering it as a service to the
>>>>>>>
>>>>>>>> service
>>>>>>>>>> registry) how can _another_ thread access the component at all?
>>>>>>>>>>
>>>>>>>>>>> I don't even see a way for components within the same DS bundle
>>>>>>>>>>> to
>>>>>>>>>>> interact with each other other than via the service registry
>>>>>>>>>>> because
>>>>>>>>>>> @Reference only works through the registry.
>>>>>>>>>>>
>>>>>>>>>>> Correction, components could interact with each other via the
>>>>>>>>>>>
>>>>>>>>>> ComponentContext.. but I guess we'd have to know if that's part of
>>>>>>>>>>
>>>>>>>>>> your
>>>>>>>>>
>>>>>>>> use
>>>>>>>
>>>>>>>> case.
>>>>>>>>>>
>>>>>>>>>> Actually, no, it's not even possible via the ComponentContext. It
>>>>>>>>>>
>>>>>>>>> only
>>>>>>>>> allows access to Services.
>>>>>>>>>
>>>>>>>>> - Ray
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please, can you explain in more detail what case might allow for
>>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> above? Because I believe the assumption is that it's not
>>>>>>>>>>> currently
>>>>>>>>>>> possible. Certainly there could be a bug and I'm certain everyone
>>>>>>>>>>>
>>>>>>>>>>> would
>>>>>>>>>>
>>>>>>>>> want to see that fixed.
>>>>>>>
>>>>>>>> --
>>>>>>>>>>> *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile>
>>>>>>>>>>> (@rotty3000)
>>>>>>>>>>> Senior Software Architect *Liferay, Inc.* <
>>>>>>>>>>> http://www.liferay.com>
>>>>>>>>>>> (@Liferay)
>>>>>>>>>>> Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org>
>>>>>>>>>>> (@OSGiAlliance)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>> *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile>
>>>>>>>>>> (@rotty3000)
>>>>>>>>>> Senior Software Architect *Liferay, Inc.* <http://www.liferay.com
>>>>>>>>>> >
>>>>>>>>>> (@Liferay)
>>>>>>>>>> Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org>
>>>>>>>>>> (@OSGiAlliance)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>> *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile>
>>>>>>>>> (@rotty3000)
>>>>>>>>> Senior Software Architect *Liferay, Inc.* <http://www.liferay.com>
>>>>>>>>> (@Liferay)
>>>>>>>>> Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org>
>>>>>>>>> (@OSGiAlliance)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>
>
