Re: Changing TCCL during deserialization

[Michał Kłeczek (XPro Sp. z o. o.)]

I am not sure how OSGI relates to this question. But I can imagine the 
situation like this:


class MySmartAssWrappingObject implements Serializable {

  Object myMember;
...

private void readObject(ObjectInputStream ois) {
  
Thread.currentThread().setContextClassLoader(getClass().getClassLoader());

  myMember = ois.readObject();
}

}

That would allow you to do something similar to what you wanted to do 
with class resolution by remembering the stack of class loaders.


So my question is:
is it something that people do?

Thanks,
Michal

Peter wrote:
  
In PreferredClassProvider, no the callers ClassLoader (context) is the parent ClassLoader of the codebase loader.


It depends on the ClassLoader hierarchy and chosen strategy used to resolve 
annotations.

But the index key for PreferrefClassProvider is  URI[] and parent loader 
(callers loader).

This strategy allows codebases to be duplicated for different calling context.

OSGi however, only loads one Bundle per URL, but as Bharath has demonstrated, 
the codebase loader doesn't have to be a BundleReference.

There are some caveats if the proxy codebase loader isn't a BundleReference, 
one is your dependencies aren't version managed for you, and you can only see 
public classes imported by the parent BundleReference.

The strategy of switching context wouldn't work with PreferredClassProvider.

Regards,

Peter.

Sent from my Samsung device.
  
   Include original message

 Original message 
From: "Michał Kłeczek (XPro Sp. z o. o.)"
Sent: 07/02/2017 07:20:59 am
To: dev@river.apache.org
Subject: Re: Changing TCCL during deserialization

This still does not answer my question - maybe I am not clear enough.
Do you have a need to set a TCCL DURING a remote call that is in progress?
Ie. you execute a remote call and DURING deserialization of the return value 
you change the TCCL (so one class is resolved using one context loader and 
another using a different one when reading THE SAME stream)

Thanks,
Michal

Gregg Wonderly wrote:
Anytime that a thread might end up being the one to download code, you need 
that threads CCL to be set.   The AWTEvent thread(s) in particular are a 
sticking point.  I have a class which I use for managing threading in 
AWT/Swing.  It’s called ComponentUpdateThread.  It works as follows.

new ComponentUpdateThread( itemList, actionButton1, actionButton2, 
checkbox1 ) {
public void setup() {
// In event thread
setBusyCursorOn( itemList );
}
public Listconstruct() {
try {
return service.getListOfItems( filterParm1 );
} catch( Exception ex ) {
reportException(ex);
}
return null;
}
public void finished() {
List  let;
if( (lst = get()) != null ) {
itemList.getModel().setContents( lst );
}
}
}.start();

This class will make the passed components disabled to keep them from being 
clicked on again, setup for processing use a non AWTEvent thread for getting 
data with other components of the UI still working, and finally mark the 
disabled components back to enabled, and load the list with the returned items, 
if there where any returned.

There is the opportunity for 3 or more threads to be involved here.  First, 
there is the calling thread.  It doesn’t do anything but start the work.  Next, 
there is an AWTEvent thread which will invoke setup().  Next there is a worker 
thread which will invoke construct().   Finally, there is (possible another) 
AWTEventThread which will invoke finished().

In total there could be up to four different threads involved, all of which 
must have TCCL set to the correct class loader.  My convention in the 
implementation, is that that will be this.getClass()getClassLoader().

This is all managed inside of the implementation of ComponentUpdateThread so 
that I don’t have to worry about it, any more.  But it’s important to 
understand that if you don’t do that, then the classes that the calling thread 
can resolve, and Item in this specific case in particular, and you would thus 
potentially see Item come from another class loader than you intended (the 
services class loader with “null” as the parent), and this will result in 
either a CNFE or CCE.

Gregg

On Feb 6, 2017, at 11:28 AM, Michał Kłeczek (XPro Sp. z o. 
o.)  wrote:

What I was specifically asking for is whether this is needed during 
deserialization or after deserialization.

In other words - if I can lock the TCCL to an instance of MarshalInputStream 
existing for the duration of a single remote call.

Thanks,
Michal

Gregg Wonderly wrote:
The predominant place where it is needed is when you download a serviceUI 
component from a proxy service which just advertises some kind of “browsing” 
interface to find specific 

Re: Changing TCCL during deserialization

[Peter]

 
In PreferredClassProvider, no the callers ClassLoader (context) is the parent 
ClassLoader of the codebase loader.

It depends on the ClassLoader hierarchy and chosen strategy used to resolve 
annotations.

But the index key for PreferrefClassProvider is  URI[] and parent loader 
(callers loader).

This strategy allows codebases to be duplicated for different calling context.

OSGi however, only loads one Bundle per URL, but as Bharath has demonstrated, 
the codebase loader doesn't have to be a BundleReference.

There are some caveats if the proxy codebase loader isn't a BundleReference, 
one is your dependencies aren't version managed for you, and you can only see 
public classes imported by the parent BundleReference.

The strategy of switching context wouldn't work with PreferredClassProvider.

Regards,

Peter.

Sent from my Samsung device.
 
  Include original message
 Original message 
From: "Michał Kłeczek (XPro Sp. z o. o.)" 
Sent: 07/02/2017 07:20:59 am
To: dev@river.apache.org
Subject: Re: Changing TCCL during deserialization

This still does not answer my question - maybe I am not clear enough.
Do you have a need to set a TCCL DURING a remote call that is in progress?
Ie. you execute a remote call and DURING deserialization of the return value 
you change the TCCL (so one class is resolved using one context loader and 
another using a different one when reading THE SAME stream)

Thanks,
Michal

Gregg Wonderly wrote:
Anytime that a thread might end up being the one to download code, you need 
that threads CCL to be set.   The AWTEvent thread(s) in particular are a 
sticking point.  I have a class which I use for managing threading in 
AWT/Swing.  It’s called ComponentUpdateThread.  It works as follows.

new ComponentUpdateThread( itemList, actionButton1, actionButton2, 
checkbox1 ) {
public void setup() {
// In event thread
setBusyCursorOn( itemList );
}
public Listconstruct() {
try {
return service.getListOfItems( filterParm1 );
} catch( Exception ex ) {
reportException(ex);
}
return null;
}
public void finished() {
List let;
if( (lst = get()) != null ) {
itemList.getModel().setContents( lst );
}
}
}.start();

This class will make the passed components disabled to keep them from being 
clicked on again, setup for processing use a non AWTEvent thread for getting 
data with other components of the UI still working, and finally mark the 
disabled components back to enabled, and load the list with the returned items, 
if there where any returned.

There is the opportunity for 3 or more threads to be involved here.  First, 
there is the calling thread.  It doesn’t do anything but start the work.  Next, 
there is an AWTEvent thread which will invoke setup().  Next there is a worker 
thread which will invoke construct().   Finally, there is (possible another) 
AWTEventThread which will invoke finished().

In total there could be up to four different threads involved, all of which 
must have TCCL set to the correct class loader.  My convention in the 
implementation, is that that will be this.getClass()getClassLoader().

This is all managed inside of the implementation of ComponentUpdateThread so 
that I don’t have to worry about it, any more.  But it’s important to 
understand that if you don’t do that, then the classes that the calling thread 
can resolve, and Item in this specific case in particular, and you would thus 
potentially see Item come from another class loader than you intended (the 
services class loader with “null” as the parent), and this will result in 
either a CNFE or CCE.

Gregg

On Feb 6, 2017, at 11:28 AM, Michał Kłeczek (XPro Sp. z o. o.) 
 wrote:

What I was specifically asking for is whether this is needed during 
deserialization or after deserialization.

In other words - if I can lock the TCCL to an instance of MarshalInputStream 
existing for the duration of a single remote call.

Thanks,
Michal

Gregg Wonderly wrote:
The predominant place where it is needed is when you download a serviceUI 
component from a proxy service which just advertises some kind of “browsing” 
interface to find specific services and interact with them, and that serviceUI 
is embedded in another application with it’s own codebase

appl->serviceUI-for-browsing->Service-to-use->That-Services-ServiceUI

In this case, TCCL must be set to the serviceui classes classloader so that the 
“serviceui-for-browsing” will have a proper parent class pointer.

Anytime that downloaded code might download more code, it should always set 
TCCL to its own class loader so that the classes it downloads reflect against 
the existing class definitions.

Gregg

On Feb 6, 2017, at 12:03 AM, Michał 

Re: Changing TCCL during deserialization

[Michał Kłeczek (XPro Sp. z o. o.)]

This still does not answer my question - maybe I am not clear enough.
Do you have a need to set a TCCL DURING a remote call that is in progress?
Ie. you execute a remote call and DURING deserialization of the return 
value you change the TCCL (so one class is resolved using one context 
loader and another using a different one when reading THE SAME stream)


Thanks,
Michal

Gregg Wonderly wrote:

Anytime that a thread might end up being the one to download code, you need 
that threads CCL to be set.   The AWTEvent thread(s) in particular are a 
sticking point.  I have a class which I use for managing threading in 
AWT/Swing.  It’s called ComponentUpdateThread.  It works as follows.

new ComponentUpdateThread( itemList, actionButton1, actionButton2, 
checkbox1 ) {
public void setup() {
// In event thread
setBusyCursorOn( itemList );
}
public Listconstruct() {
try {
return service.getListOfItems( filterParm1 );
} catch( Exception ex ) {
reportException(ex);
}
return null;
}
public void finished() {
List  let;
if( (lst = get()) != null ) {
itemList.getModel().setContents( lst );
}
}
}.start();

This class will make the passed components disabled to keep them from being 
clicked on again, setup for processing use a non AWTEvent thread for getting 
data with other components of the UI still working, and finally mark the 
disabled components back to enabled, and load the list with the returned items, 
if there where any returned.

There is the opportunity for 3 or more threads to be involved here.  First, 
there is the calling thread.  It doesn’t do anything but start the work.  Next, 
there is an AWTEvent thread which will invoke setup().  Next there is a worker 
thread which will invoke construct().   Finally, there is (possible another) 
AWTEventThread which will invoke finished().

In total there could be up to four different threads involved, all of which 
must have TCCL set to the correct class loader.  My convention in the 
implementation, is that that will be this.getClass().getClassLoader().

This is all managed inside of the implementation of ComponentUpdateThread so 
that I don’t have to worry about it, any more.  But it’s important to 
understand that if you don’t do that, then the classes that the calling thread 
can resolve, and Item in this specific case in particular, and you would thus 
potentially see Item come from another class loader than you intended (the 
services class loader with “null” as the parent), and this will result in 
either a CNFE or CCE.

Gregg


On Feb 6, 2017, at 11:28 AM, Michał Kłeczek (XPro Sp. z o. 
o.)  wrote:

What I was specifically asking for is whether this is needed during 
deserialization or after deserialization.

In other words - if I can lock the TCCL to an instance of MarshalInputStream 
existing for the duration of a single remote call.

Thanks,
Michal

Gregg Wonderly wrote:

The predominant place where it is needed is when you download a serviceUI 
component from a proxy service which just advertises some kind of “browsing” 
interface to find specific services and interact with them, and that serviceUI 
is embedded in another application with it’s own codebase

appl->serviceUI-for-browsing->Service-to-use->That-Services-ServiceUI

In this case, TCCL must be set to the serviceui classes classloader so that the 
“serviceui-for-browsing” will have a proper parent class pointer.

Anytime that downloaded code might download more code, it should always set 
TCCL to its own class loader so that the classes it downloads reflect against 
the existing class definitions.

Gregg


On Feb 6, 2017, at 12:03 AM, Michał Kłeczek (XPro Sp. z o. 
o.)    wrote:

Hi,

During my work on object based annotations I realized it would be more efficient not to 
look for TCCL upon every call to "load class" (when default loader does not 
match the annotation).
It might be more effective to look it up upon stream creation and using it 
subsequently for class loader selection.

But this might change semantics of deserialization a little bit - it would not 
be possible to change the context loader during deserialization.
My question is - are there any scenarios that require that?
I cannot think of any but...

Thanks,
Michal

Re: Changing TCCL during deserialization

[Gregg Wonderly]

Anytime that a thread might end up being the one to download code, you need 
that threads CCL to be set.   The AWTEvent thread(s) in particular are a 
sticking point.  I have a class which I use for managing threading in 
AWT/Swing.  It’s called ComponentUpdateThread.  It works as follows.

new ComponentUpdateThread( itemList, actionButton1, actionButton2, 
checkbox1 ) {
public void setup() {
// In event thread
setBusyCursorOn( itemList );
}
public Listconstruct() {
try {
return service.getListOfItems( filterParm1 );
} catch( Exception ex ) {
reportException(ex);
}
return null;
}
public void finished() {
List let;
if( (lst = get()) != null ) {
itemList.getModel().setContents( lst );
}
}
}.start();

This class will make the passed components disabled to keep them from being 
clicked on again, setup for processing use a non AWTEvent thread for getting 
data with other components of the UI still working, and finally mark the 
disabled components back to enabled, and load the list with the returned items, 
if there where any returned.

There is the opportunity for 3 or more threads to be involved here.  First, 
there is the calling thread.  It doesn’t do anything but start the work.  Next, 
there is an AWTEvent thread which will invoke setup().  Next there is a worker 
thread which will invoke construct().   Finally, there is (possible another) 
AWTEventThread which will invoke finished().

In total there could be up to four different threads involved, all of which 
must have TCCL set to the correct class loader.  My convention in the 
implementation, is that that will be this.getClass().getClassLoader().

This is all managed inside of the implementation of ComponentUpdateThread so 
that I don’t have to worry about it, any more.  But it’s important to 
understand that if you don’t do that, then the classes that the calling thread 
can resolve, and Item in this specific case in particular, and you would thus 
potentially see Item come from another class loader than you intended (the 
services class loader with “null” as the parent), and this will result in 
either a CNFE or CCE.

Gregg

> On Feb 6, 2017, at 11:28 AM, Michał Kłeczek (XPro Sp. z o. o.) 
>  wrote:
> 
> What I was specifically asking for is whether this is needed during 
> deserialization or after deserialization.
> 
> In other words - if I can lock the TCCL to an instance of MarshalInputStream 
> existing for the duration of a single remote call.
> 
> Thanks,
> Michal
> 
> Gregg Wonderly wrote:
>> The predominant place where it is needed is when you download a serviceUI 
>> component from a proxy service which just advertises some kind of “browsing” 
>> interface to find specific services and interact with them, and that 
>> serviceUI is embedded in another application with it’s own codebase
>> 
>> appl->serviceUI-for-browsing->Service-to-use->That-Services-ServiceUI
>> 
>> In this case, TCCL must be set to the serviceui classes classloader so that 
>> the “serviceui-for-browsing” will have a proper parent class pointer.
>> 
>> Anytime that downloaded code might download more code, it should always set 
>> TCCL to its own class loader so that the classes it downloads reflect 
>> against the existing class definitions.
>> 
>> Gregg
>> 
>>> On Feb 6, 2017, at 12:03 AM, Michał Kłeczek (XPro Sp. z o. o.) 
>>>   wrote:
>>> 
>>> Hi,
>>> 
>>> During my work on object based annotations I realized it would be more 
>>> efficient not to look for TCCL upon every call to "load class" (when 
>>> default loader does not match the annotation).
>>> It might be more effective to look it up upon stream creation and using it 
>>> subsequently for class loader selection.
>>> 
>>> But this might change semantics of deserialization a little bit - it would 
>>> not be possible to change the context loader during deserialization.
>>> My question is - are there any scenarios that require that?
>>> I cannot think of any but...
>>> 
>>> Thanks,
>>> Michal
>> 
> 

Re: Draft of February 2017 report

[Bryan Thompson]

+1

Bryan

On Feb 6, 2017 9:25 AM, "Patricia Shanahan"  wrote:

> +1
>
> Active PMC members: Please approve or comment.
>
> On 2/4/2017 9:35 AM, Patricia Shanahan wrote:
>
>> ## Description:
>>
>>  - Apache River software provides a standards-compliant JINI service.
>>
>> ## Issues:
>>
>>  - There are no issues requiring board attention at this time.
>>
>> ## Activity:
>>
>>  - Continued discussion of River's future direction, this quarter mainly
>> on OSGi and serialization.
>>
>>  - Zsolt Kúti reworked the River web site.
>>
>> ## Health report:
>>
>>  - The future directions discussion continues.
>>
>>  - Attracting new developers will remain difficult until the future
>> direction is firmed up and made visible. We did add one committer this
>> quarter.
>>
>> ## PMC changes:
>>
>>  - Currently 11 PMC members.
>>  - No new PMC members added in the last 3 months
>>  - Last PMC addition was Bryan Thompson on Sun Aug 30 2015
>>
>> ## Committer base changes:
>>
>>  - Currently 14 committers.
>>  - Zsolt Kúti was added as a committer on Wed Dec 07 2016
>>
>> ## Releases:
>>
>>  - River-3.0.0 was released on Wed Oct 05 2016
>>
>> ## Mailing list activity:
>>
>>  - There has been a recent increase in dev@ activity due to a lively
>> technical discussion of OSGi and serialization in the context of River.
>> That discussion is on-going.
>>
>>  - dev@river.apache.org:
>> - 95 subscribers (up 0 in the last 3 months):
>> - 201 emails sent to list (114 in previous quarter)
>>
>>  - u...@river.apache.org:
>> - 96 subscribers (up 0 in the last 3 months):
>> - 2 emails sent to list (3 in previous quarter)
>>
>>
>> ## JIRA activity:
>>
>>  - 2 JIRA tickets created in the last 3 months
>>  - 1 JIRA tickets closed/resolved in the last 3 months
>>
>

Re: Changing TCCL during deserialization

[Michał Kłeczek (XPro Sp. z o. o.)]

What I was specifically asking for is whether this is needed during 
deserialization or after deserialization.


In other words - if I can lock the TCCL to an instance of 
MarshalInputStream existing for the duration of a single remote call.


Thanks,
Michal

Gregg Wonderly wrote:

The predominant place where it is needed is when you download a serviceUI 
component from a proxy service which just advertises some kind of “browsing” 
interface to find specific services and interact with them, and that serviceUI 
is embedded in another application with it’s own codebase

appl->serviceUI-for-browsing->Service-to-use->That-Services-ServiceUI

In this case, TCCL must be set to the serviceui classes classloader so that the 
“serviceui-for-browsing” will have a proper parent class pointer.

Anytime that downloaded code might download more code, it should always set 
TCCL to its own class loader so that the classes it downloads reflect against 
the existing class definitions.

Gregg


On Feb 6, 2017, at 12:03 AM, Michał Kłeczek (XPro Sp. z o. 
o.)  wrote:

Hi,

During my work on object based annotations I realized it would be more efficient not to 
look for TCCL upon every call to "load class" (when default loader does not 
match the annotation).
It might be more effective to look it up upon stream creation and using it 
subsequently for class loader selection.

But this might change semantics of deserialization a little bit - it would not 
be possible to change the context loader during deserialization.
My question is - are there any scenarios that require that?
I cannot think of any but...

Thanks,
Michal

Re: Draft of February 2017 report

[Patricia Shanahan]

+1

Active PMC members: Please approve or comment.

On 2/4/2017 9:35 AM, Patricia Shanahan wrote:

## Description:

 - Apache River software provides a standards-compliant JINI service.

## Issues:

 - There are no issues requiring board attention at this time.

## Activity:

 - Continued discussion of River's future direction, this quarter mainly
on OSGi and serialization.

 - Zsolt Kúti reworked the River web site.

## Health report:

 - The future directions discussion continues.

 - Attracting new developers will remain difficult until the future
direction is firmed up and made visible. We did add one committer this
quarter.

## PMC changes:

 - Currently 11 PMC members.
 - No new PMC members added in the last 3 months
 - Last PMC addition was Bryan Thompson on Sun Aug 30 2015

## Committer base changes:

 - Currently 14 committers.
 - Zsolt Kúti was added as a committer on Wed Dec 07 2016

## Releases:

 - River-3.0.0 was released on Wed Oct 05 2016

## Mailing list activity:

 - There has been a recent increase in dev@ activity due to a lively
technical discussion of OSGi and serialization in the context of River.
That discussion is on-going.

 - dev@river.apache.org:
- 95 subscribers (up 0 in the last 3 months):
- 201 emails sent to list (114 in previous quarter)

 - u...@river.apache.org:
- 96 subscribers (up 0 in the last 3 months):
- 2 emails sent to list (3 in previous quarter)


## JIRA activity:

 - 2 JIRA tickets created in the last 3 months
 - 1 JIRA tickets closed/resolved in the last 3 months

Re: OSGi

[Gregg Wonderly]

I still feel that RMIClassLoaderSPI can provide this mechanism.  There is an 
execution context required, but predominately, the URL string can still reflect 
the source of the code you want to use.

Gregg

> On Feb 5, 2017, at 11:34 PM, Michał Kłeczek (XPro Sp. z o. o.) 
>  wrote:
> 
> Once you realize you need some codebase metadata different than mere list of 
> URLs
> the next conclusion is that annotations should be something different than... 
> a String :)
> 
> The next thing to ask is: "what about mixed OSGI and non-OSGI environments"
> Then you start to realize you need to abstract over the class loading 
> environment itself.
> 
> Then you start to realize that to support all the scenarios you need to 
> provide a class loading environment that is "pluggable"
> - ie allows using it with other class loading environments and allow the user 
> to decide which classes should be loaded
> by which environment.
> 
> This is what I am working on right now :)
> 
> Thanks,
> Michal
> 
> Peter wrote:
>> My phone sent the previous email before I completed editing.
>> 
>> ...If api classes are already loaded locally by client code, then a smart 
>> proxy codebase bundle will resolve imports to those packages (if they're 
>> within the imported version range), when the proxy bundle is downloaded, 
>> resolved and loaded.
>> 
>> The strategy should be, deserialize using the callers context until a class 
>> is not found, then switch to the object containing the current field being 
>> deserialized (which may be a package private implementation class in the 
>> service api bundle) and if that fails use the codebase annotation (the smart 
>> proxy).  This is similar in some ways to never preferred, where locally 
>> visible classes will be selected first.
>> 
>> The strategy is to let OSGi do all the dependency wiring from bundle 
>> manifests.  Classes not visible will be visible from a common package import 
>> class, except for poorly designed services, which is outside of scope.
>> 
>> Only match api version compatible services.
>> 
>> No allowances made for split packages or other complexities.
>> 
>> If deserialization doesn't succeed, look up another service.
>> 
>> Cheers,
>> 
>> Peter.
>> 
>> Sent from my Samsung device.
>> Include original message
>>  Original message 
>> From: Peter
>> Sent: 06/02/2017 02:59:09 pm
>> To: dev@river.apache.org
>> Subject: Re: OSGi
>> 
>>  Thanks Nic,
>> 
>> If annot
>> 
>> You've identified the reason we need an OSGi specific RMIClassLoaderSpi 
>> implementation; so we can capture and provide Bundle specific annotation 
>> information.
>> 
>> Rmiclassloaderspi's loadClass method expects a ClassLoader to be passed in, 
>> the context ClassLoader is used by PreferredClassProvider when the 
>> ClassLoader argument is null.
>> 
>> Standard Java serialization's OIS walks the call stack and selects the first 
>> non system classloader (it's looking for the application class loader), it 
>> deserializes into the application ClassLoader's context.  This doesn't  work 
>> in OSGi because the application classes are loaded by a multitude of 
>> ClassLoaders.
>> 
>> It also looks like we'll need an OSGi specific InvocationLayerFactory to 
>> capture ClassLoader information to pass to our MarshalInputStream then to 
>> our RMIClassLoaderSpi during deserialization at both endpoints.
>> 
>> We also need to know the bundle (ClassLoader) of the class that calls a 
>> java.lang.reflect.Proxy on the client side, this is actually quite easy to 
>> find, walk the stack, find the Proxy class and obtain the BundleReference / 
>> ClassLoader of the caller.
>> 
>> Currently the java.lang.reflectProxy dynamically generated subclass instance 
>> proxy's ClassLoader is used, this is acceptable when the proxy bytecode is 
>> loaded by the the Client's ClassLoader or smart proxy ClassLoader in the 
>> case where a smart proxy is utilised
>> 
>> 
>> 
>> If the caller changes, so does the calling context.
>> 
>> 
>> Each bundle provides access to all classes within that bundle, including any 
>> public classes from imported packages.
>> 
>> 
>> 
>> 
>> 
>> Sent from my Samsung device.
>> Include original message
>>  Original message 
>> From: Niclas Hedhman
>> Sent: 04/02/2017 12:43:28 pm
>> To: dev@river.apache.org
>> Subject: Re: OSGi
>> 
>> 
>> 
>> Further, I think the only "sane" approach in a OSGi environment is to create 
>> a new bundle for the Remote environment, all codebases not part of the API 
>> goes into that bundle and that the API is required to be present in the OSGi 
>> environment a priori. I.e. treat the Remote objects in OSGi as it is treated 
>> in plain Java; one classloader, one chunk, sort out its own serialization 
>> woes. Likewise for the server; treat it as ordinary RMI, without any 
>> mumbo-jambo OSGi stuff to be figured out at a non-OSGi-running JVM. An 
>> 

Re: Changing TCCL during deserialization

[Gregg Wonderly]

The predominant place where it is needed is when you download a serviceUI 
component from a proxy service which just advertises some kind of “browsing” 
interface to find specific services and interact with them, and that serviceUI 
is embedded in another application with it’s own codebase

appl->serviceUI-for-browsing->Service-to-use->That-Services-ServiceUI

In this case, TCCL must be set to the serviceui classes classloader so that the 
“serviceui-for-browsing” will have a proper parent class pointer.

Anytime that downloaded code might download more code, it should always set 
TCCL to its own class loader so that the classes it downloads reflect against 
the existing class definitions.

Gregg

> On Feb 6, 2017, at 12:03 AM, Michał Kłeczek (XPro Sp. z o. o.) 
>  wrote:
> 
> Hi,
> 
> During my work on object based annotations I realized it would be more 
> efficient not to look for TCCL upon every call to "load class" (when default 
> loader does not match the annotation).
> It might be more effective to look it up upon stream creation and using it 
> subsequently for class loader selection.
> 
> But this might change semantics of deserialization a little bit - it would 
> not be possible to change the context loader during deserialization.
> My question is - are there any scenarios that require that?
> I cannot think of any but...
> 
> Thanks,
> Michal

Re: AbstractILFactory bug?

[Peter]

Hmm well spotted, better report that one :)

Cheers,

Peter.

Sent from my Samsung device.
 
  Include original message
 Original message 
From: "Michał Kłeczek (XPro Sp. z o. o.)" 
Sent: 06/02/2017 07:51:20 pm
To: dev@river.apache.org
Subject: Re: AbstractILFactory bug?

I'm talking about this: 
Util.checkPackageAccess(interfaces[i].getClass()); //NOTE the getClass()  
here!!! 

It should be: 
Util.checkPackageAccess(interfaces[i]); 

Michal 

Michał Kłeczek (XPro Sp. z o. o.) wrote: 
> I understand the check is needed. 
> 
> It is that we are not checking the right package but "java.lang" 
> 
> Thanks, 
> Michal 
> 
> Peter wrote: 
>> Ok, worked out why, java.lang.reflect.Proxy's newProxyInstance  
>> permission check  is caller sensitive.  In this case  
>> AbstractILFactory is the caller, so not checking it would allow an  
>> attacker to bypass the check using AbstractILFactory. 
>> Cheers, 
>> 
>> Peter. 
>> 
>> Sent from my Samsung device. 
>>  Include original message 
>>  Original message  
>> From: "Michał Kłeczek (XPro Sp. z o. o.)" 
>> Sent: 06/02/2017 05:06:32 pm 
>> To: dev@river.apache.org 
>> Subject: AbstractILFactory bug? 
>> 
>> I have just found this piece of code in AbstractILFactory: 
>> 
>> Class[] interfaces = getProxyInterfaces(impl); 
>> ... 
>> for (int i = 0; i<  interfaces.length; i++) { 
>>   Util.checkPackageAccess(interfaces[i].getClass()); 
>> } 
>> 
>> So we check "java.lang" package access. 
>> 
>> A bug? 
>> 
>> Thanks, 
>> Michal 
>> 
>> 
> 

Re: AbstractILFactory bug?

[Michał Kłeczek (XPro Sp. z o. o.)]

I'm talking about this:
Util.checkPackageAccess(interfaces[i].getClass()); //NOTE the getClass() 
here!!!


It should be:
Util.checkPackageAccess(interfaces[i]);

Michal

Michał Kłeczek (XPro Sp. z o. o.) wrote:

I understand the check is needed.

It is that we are not checking the right package but "java.lang"

Thanks,
Michal

Peter wrote:
Ok, worked out why, java.lang.reflect.Proxy's newProxyInstance 
permission check  is caller sensitive.  In this case 
AbstractILFactory is the caller, so not checking it would allow an 
attacker to bypass the check using AbstractILFactory.

Cheers,

Peter.

Sent from my Samsung device.
 Include original message
 Original message 
From: "Michał Kłeczek (XPro Sp. z o. o.)"
Sent: 06/02/2017 05:06:32 pm
To: dev@river.apache.org
Subject: AbstractILFactory bug?

I have just found this piece of code in AbstractILFactory:

Class[] interfaces = getProxyInterfaces(impl);
...
for (int i = 0; i<  interfaces.length; i++) {
  Util.checkPackageAccess(interfaces[i].getClass());
}

So we check "java.lang" package access.

A bug?

Thanks,
Michal

Re: AbstractILFactory bug?

[Michał Kłeczek (XPro Sp. z o. o.)]

I understand the check is needed.

It is that we are not checking the right package but "java.lang"

Thanks,
Michal

Peter wrote:
Ok, worked out why, java.lang.reflect.Proxy's newProxyInstance permission check  is caller sensitive.  In this case AbstractILFactory is the caller, so not checking it would allow an attacker to bypass the check using AbstractILFactory. 


Cheers,

Peter.

Sent from my Samsung device.
  
   Include original message

 Original message 
From: "Michał Kłeczek (XPro Sp. z o. o.)"
Sent: 06/02/2017 05:06:32 pm
To: dev@river.apache.org
Subject: AbstractILFactory bug?

I have just found this piece of code in AbstractILFactory:

Class[] interfaces = getProxyInterfaces(impl);
...
for (int i = 0; i<  interfaces.length; i++) {
  Util.checkPackageAccess(interfaces[i].getClass());
}

So we check "java.lang" package access.

A bug?

Thanks,
Michal

Re: AbstractILFactory bug?

[Peter]

Ok, worked out why, java.lang.reflect.Proxy's newProxyInstance permission check 
 is caller sensitive.  In this case AbstractILFactory is the caller, so not 
checking it would allow an attacker to bypass the check using 
AbstractILFactory. 

Cheers,

Peter.

Sent from my Samsung device.
 
  Include original message
 Original message 
From: "Michał Kłeczek (XPro Sp. z o. o.)" 
Sent: 06/02/2017 05:06:32 pm
To: dev@river.apache.org
Subject: AbstractILFactory bug?

I have just found this piece of code in AbstractILFactory: 

Class[] interfaces = getProxyInterfaces(impl); 
... 
for (int i = 0; i < interfaces.length; i++) { 
 Util.checkPackageAccess(interfaces[i].getClass()); 
} 

So we check "java.lang" package access. 

A bug? 

Thanks, 
Michal 

Re: OSGi

[Michał Kłeczek (XPro Sp. z o. o.)]

The upside is that it simplifies the overall architecture.
For example it makes the whole part of River related to proxy trust 
verification obsolete.
All these ProxyTrustIterators executed in an untrusted security context, 
Verifier implementations loaded using a proper ClassLoader etc. - this 
is not needed anymore.


Thanks,
Michal

Michał Kłeczek (XPro Sp. z o. o.) wrote:

Well - times changed since original Jini has been developed.
There is a whole lot of amazing libraries out there - so the 
undertaking is much easier than doing it without them.

I am specifically talking about Google Guava, JBoss Modules and RxJava.

As River is concerned - once you get past the assumption that codebase 
annotations are Strings - it has all the necessary extension points 
available.


I've already started writing the test suite for the thing and hope to 
present it soon.


Thanks,
Michal

Peter wrote:
For the sake of simplicity it's probably best if OSGi and non interact only using reflection proxy's and have their own djinn groups so code downloading is unnecessary between them. 


At least that's how I'd consider introducing it into an existing djinn.

A jvm that doesn't have version management of some sort may have a lot of 
difficulty interacting with services from a framework that can use incompatible 
library versions (and that includes service api) side by side.

My concern is interacting with non versioned env's will probably cause the 
developer to have to continue dealing with the problems the modular framework 
they selected intended solving

Maven and OSGi can probably interact using mvn: codebase annotations, provided 
all modules have bundle manifests.

I still support what your doing and find it interesting and don't wish to 
discourage you, I think you're likely to admit it will be a difficult 
undertaking, but that's probably an attraction right?  Maybe River could 
provide some interfaces for extensibility where you could plug in?

Regards,

Peter.

Sent from my Samsung device.
  
   Include original message

 Original message 
From: "Michał Kłeczek (XPro Sp. z o. o.)"
Sent: 06/02/2017 03:34:54 pm
To:dev@river.apache.org
Subject: Re: OSGi

Once you realize you need some codebase metadata different than mere 
list of URLs
the next conclusion is that annotations should be something different 
than... a String :)


The next thing to ask is: "what about mixed OSGI and non-OSGI environments"
Then you start to realize you need to abstract over the class loading 
environment itself.


Then you start to realize that to support all the scenarios you need to 
provide a class loading environment that is "pluggable"
- ie allows using it with other class loading environments and allow the 
user to decide which classes should be loaded

by which environment.

This is what I am working on right now :)

Thanks,
Michal

Peter wrote:

  My phone sent the previous email before I completed editing.

  ...If api classes are already loaded locally by client code, then a smart 
proxy codebase bundle will resolve imports to those packages (if they're within 
the imported version range), when the proxy bundle is downloaded, resolved and 
loaded.

  The strategy should be, deserialize using the callers context until a class 
is not found, then switch to the object containing the current field being 
deserialized (which may be a package private implementation class in the 
service api bundle) and if that fails use the codebase annotation (the smart 
proxy).  This is similar in some ways to never preferred, where locally visible 
classes will be selected first.

  The strategy is to let OSGi do all the dependency wiring from bundle 
manifests.  Classes not visible will be visible from a common package import 
class, except for poorly designed services, which is outside of scope.

  Only match api version compatible services.

  No allowances made for split packages or other complexities.

  If deserialization doesn't succeed, look up another service.

  Cheers,

  Peter.

  Sent from my Samsung device.

 Include original message

   Original message 
  From: Peter
  Sent: 06/02/2017 02:59:09 pm
  To:dev@river.apache.org
  Subject: Re: OSGi


  Thanks Nic,


  If annot

  You've identified the reason we need an OSGi specific RMIClassLoaderSpi 
implementation; so we can capture and provide Bundle specific annotation 
information.

  Rmiclassloaderspi's loadClass method expects a ClassLoader to be passed in, 
the context ClassLoader is used by PreferredClassProvider when the ClassLoader 
argument is null.

  Standard Java serialization's OIS walks the call stack and selects the first 
non system classloader (it's looking for the application class loader), it 
deserializes into the application ClassLoader's context.  This doesn't  work in 
OSGi because the application classes are loaded by a multitude of ClassLoaders.

  It also looks