Re: Issues not closed for 2.3.0 release

[Marshall Schor]

Marshall Schor wrote:
> Here's my current better(?) thought that avoids injecting and doesn't
> presume the SystemClassLoader is a subclass of URLClassLoader, and
> avoids permission issues that could arise in some environments when
> later adding to the URL set via addURL():
>
> Use the defined property
> -Djava.system.class.loader=org.apache.uima.bootstrap.SystemClassLoader
>
> and add a class SystemClassLoader which is defined to be a subclass of
> URL ClassLoader, and delegates to it.  Define the constructor
> SystemClassLoader(ClassLoader parent) to call its parent with the proper
> list of URLs as before.
>
> No reflection is needed.
>
> Now to get this tested :-) 

Testing: works on Sun Java, fails on IBM Java.  This is because the IBM
java doesn't load the logger-specified classes from the
ClassLoader.getSystemClassLoader() object, even though the Javadocs say
so. 

I'm going back to using "injection" approach, that at least works (has
been tested) on both Javas.

-Marshall
>  -Marshall
>
> Marshall Schor wrote:
>   
>> Marshall Schor wrote:
>>   
>> 
>>> Thilo Goetz wrote:
>>>   
>>> 
>>>   
 Marshall Schor wrote:
 [...]
   
 
   
 
> Both of these approaches put uimaj-core classes at the "root" level of
> the application (i.e., "System") class-loader chain.  I can't think of
> any cases where that would not be OK; can anyone else?
> 
>   
> 
>   
 I think that may not work because the UIMA
 framework classes need to access the annotator classes.
 If the core is loaded by the system classloader, it
 will not have access to classes loaded by the bs loader.
   
 
   
 
>>> Yes, this sounds right.
>>>
>>> How about this "fix": put a copy of the UIMALogFormatter into the
>>> bootstrap loader Jar.  This class does not reference any uima classes,
>>> only Java JRE classes.
>>>   
>>> 
>>>   
>> I think this issue is general, not restricted to the Formatter class. 
>> The Javadocs for the LogManager say that the class objects named in the
>> configuration are loaded by using the System class loader.  It actually
>> says the classes are "first searched on the system class path before any
>> user class path."   But it seems that only the application initial class
>> path that is used to launch things, is searched.
>>
>> So maybe the only approach will be to inject all the urls into the
>> system class loader, as Jerry was trying to do (Jars and directories,
>> not individual classes). 
>>
>> Any other ideas?
>>
>> -Marshall
>>   
>> 
>>> -Marshall
>>>
>>>   
>>> 
>>>   
 --Thilo



   
 
   
 
>>>   
>>> 
>>>   
>>   
>> 
>
>
>   

Re: Issues not closed for 2.3.0 release

[Marshall Schor]

Here's my current better(?) thought that avoids injecting and doesn't
presume the SystemClassLoader is a subclass of URLClassLoader, and
avoids permission issues that could arise in some environments when
later adding to the URL set via addURL():

Use the defined property
-Djava.system.class.loader=org.apache.uima.bootstrap.SystemClassLoader

and add a class SystemClassLoader which is defined to be a subclass of
URL ClassLoader, and delegates to it.  Define the constructor
SystemClassLoader(ClassLoader parent) to call its parent with the proper
list of URLs as before.

No reflection is needed.

Now to get this tested :-)  -Marshall

Marshall Schor wrote:
> Marshall Schor wrote:
>   
>> Thilo Goetz wrote:
>>   
>> 
>>> Marshall Schor wrote:
>>> [...]
>>>   
>>> 
>>>   
 Both of these approaches put uimaj-core classes at the "root" level of
 the application (i.e., "System") class-loader chain.  I can't think of
 any cases where that would not be OK; can anyone else?
 
   
 
>>> I think that may not work because the UIMA
>>> framework classes need to access the annotator classes.
>>> If the core is loaded by the system classloader, it
>>> will not have access to classes loaded by the bs loader.
>>>   
>>> 
>>>   
>> Yes, this sounds right.
>>
>> How about this "fix": put a copy of the UIMALogFormatter into the
>> bootstrap loader Jar.  This class does not reference any uima classes,
>> only Java JRE classes.
>>   
>> 
> I think this issue is general, not restricted to the Formatter class. 
> The Javadocs for the LogManager say that the class objects named in the
> configuration are loaded by using the System class loader.  It actually
> says the classes are "first searched on the system class path before any
> user class path."   But it seems that only the application initial class
> path that is used to launch things, is searched.
>
> So maybe the only approach will be to inject all the urls into the
> system class loader, as Jerry was trying to do (Jars and directories,
> not individual classes). 
>
> Any other ideas?
>
> -Marshall
>   
>> -Marshall
>>
>>   
>> 
>>> --Thilo
>>>
>>>
>>>
>>>   
>>> 
>>>   
>>   
>> 
>
>
>   

Re: Issues not closed for 2.3.0 release

[Marshall Schor]

Marshall Schor wrote:
> Thilo Goetz wrote:
>   
>> Marshall Schor wrote:
>> [...]
>>   
>> 
>>> Both of these approaches put uimaj-core classes at the "root" level of
>>> the application (i.e., "System") class-loader chain.  I can't think of
>>> any cases where that would not be OK; can anyone else?
>>> 
>>>   
>> I think that may not work because the UIMA
>> framework classes need to access the annotator classes.
>> If the core is loaded by the system classloader, it
>> will not have access to classes loaded by the bs loader.
>>   
>> 
> Yes, this sounds right.
>
> How about this "fix": put a copy of the UIMALogFormatter into the
> bootstrap loader Jar.  This class does not reference any uima classes,
> only Java JRE classes.
>   
I think this issue is general, not restricted to the Formatter class. 
The Javadocs for the LogManager say that the class objects named in the
configuration are loaded by using the System class loader.  It actually
says the classes are "first searched on the system class path before any
user class path."   But it seems that only the application initial class
path that is used to launch things, is searched.

So maybe the only approach will be to inject all the urls into the
system class loader, as Jerry was trying to do (Jars and directories,
not individual classes). 

Any other ideas?

-Marshall
> -Marshall
>
>   
>> --Thilo
>>
>>
>>
>>   
>> 
>
>
>   

Re: Issues not closed for 2.3.0 release

[Marshall Schor]

Thilo Goetz wrote:
> Marshall Schor wrote:
> [...]
>   
>> Both of these approaches put uimaj-core classes at the "root" level of
>> the application (i.e., "System") class-loader chain.  I can't think of
>> any cases where that would not be OK; can anyone else?
>> 
>
> I think that may not work because the UIMA
> framework classes need to access the annotator classes.
> If the core is loaded by the system classloader, it
> will not have access to classes loaded by the bs loader.
>   
Yes, this sounds right.

How about this "fix": put a copy of the UIMALogFormatter into the
bootstrap loader Jar.  This class does not reference any uima classes,
only Java JRE classes.

-Marshall

> --Thilo
>
>
>
>   

Re: Issues not closed for 2.3.0 release

[Thilo Goetz]

Marshall Schor wrote:
[...]
> Both of these approaches put uimaj-core classes at the "root" level of
> the application (i.e., "System") class-loader chain.  I can't think of
> any cases where that would not be OK; can anyone else?

I think that may not work because the UIMA
framework classes need to access the annotator classes.
If the core is loaded by the system classloader, it
will not have access to classes loaded by the bs loader.

--Thilo

Re: Issues not closed for 2.3.0 release

[Marshall Schor]

The basic issue seems to be that part of the initialization sequence for
logging is not finding a handler class.

The basic "fix" for this seems to be to arrange that the class for the
handler be found in the initial application class path.  This is what
the "System" class path seems to mean (taken from the Javadocs for
ClassLoader.getSystemClassLoader()). 

I noticed that the IBM Java loads this class slightly differently than
the Java Jerry described.  I didn't find a "spec" for this, so I suspect
different Javas may load this in different ways.  The IBM Java 5 uses
the last non-system class loader on the call stack.  This might be a
fine distinction, coming into play only when loading from the
"extensions" classes.

A general solution with respect to using the bootstrap loader would be
to not use the java -jar xxx form but instead use the java -cp 
org.apache.uima.bootstrap.UimaBootstrap etc. form, and put into the -cp
classpath the uima-core jar, where the handler(s) are, along with the
bootstrap jar.

Another alternative would be to build the UimaBootstrap jar so that it
included uimaj-core, or vice-versa. (I think that would work - but
haven't test it).  I'm not sure of all the implications of this, though. 

Putting the bootstrap class into uimaj-core, and setting the manifest to
specify starting with the bootstrap, and then changing the start to java
-jar uima-core.jar, is (in more detail) what Burn was suggesting below,
I think.

Both of these approaches put uimaj-core classes at the "root" level of
the application (i.e., "System") class-loader chain.  I can't think of
any cases where that would not be OK; can anyone else?

-Marshall

Burn Lewis wrote:
> Can we put the bootstrap class in uima-core so it will then have all those
> classes in the system path.
>
> - Burn.
>
>   

Re: Issues not closed for 2.3.0 release

[Burn Lewis]

Can we put the bootstrap class in uima-core so it will then have all those
classes in the system path.

- Burn.

Re: Issues not closed for 2.3.0 release

[Thilo Goetz]

Jaroslaw Cwiklik wrote:
> Marshall, I cant close https://issues.apache.org/jira/browse/UIMA-1531 yet.
> Over the weekend I've discovered why the Logger ignores uima
> formatter defined in  Logger.properties and logs entries using the default
> formatter (xml). It turns out that the java LogManager uses system
> classloader to load the formatter class. Here is the method where this magic
> occurs:
> Formatter getFormatterProperty(String name, Formatter defaultValue) {
> String val = getProperty(name);
> try {
> if (val != null) {
> Class clz = ClassLoader.getSystemClassLoader().loadClass(val);
> return (Formatter) clz.newInstance();
> }
> } catch (Exception ex) {
> // We got one of a variety of exceptions in creating the
> // class or creating an instance.
> // Drop through.
> }
> // We got an exception. Return the defaultValue.
> return defaultValue;
> }
> 
> It surprising that that this java component silently ignores errors. Since
> the uima formatter class is not loaded into the system classloader by the
> UimaBootstrap, the LogManager returns a default formatter from the method
> above.
> 
> I dont know how to fix this problem yet. One possibility is to change
> UimaBootstrap to load jars/classes into the system classloader. Another
> possibility is to load this one class into the system classloader although
> I've failed trying to do this. I attempted to augment
> the system classloader using reflection to inject a new URL like so:
> 
>   URL url = new
> URL("jar:file:/C:/uima/releases/2.3.3-04/apache-uima/lib/uima-core.jar!/java/util/logging/XMLFormatter.class");
> 
> Class[] parameters = new Class[]{URL.class};
> URLClassLoader sysloader = (URLClassLoader)
> ClassLoader.getSystemClassLoader();
> Class sysclass = URLClassLoader.class;
> try {
>Method method = sysclass.getDeclaredMethod("addURL", parameters);
>method.setAccessible(true);
>method.invoke(sysloader, new Object[]{url});
> } catch (Throwable t) {
>t.printStackTrace();
>throw new IOException("Error, could not add URL to system
> classloader");
> }//end try catch
> 
> Again, for some reason this doesnt work.I can inject jar files into the
> system classloader using the above technique but not individual classes from
> jars.

Because the system classloader only understands jars
and directories.  You can load the class file into
memory and feed the byte[] to the classloader with
the same kind of reflection techniques you use above.

Note that I'm _not_ recommending this, simply pointing
it out.

--Thilo

> 
> 
> Jerry
> 
> 
> On Sun, Oct 18, 2009 at 4:22 PM, Marshall Schor  wrote:
> 
>> Hi everyone,
>>
>> I'm "itching" to get the 2.3.0 release done.
>>
>> Of the issues that are open, reopened, or in-progress, most are now
>> being deferred to past this release.
>>
>> The ones left that may need to be worked on before the release is done
>> include:
>>
>> (Joern) UIMA-1619 
>> Clarify that a Cas Editor Project is required to use the Cas Editor in
>> the documentation 
>> (Eddie)UIMA-1611 
>> UimacppServiceControlled need to implement isStopped()
>> 
>> (Marshall) UIMA-1539 
>> Update READMEs for 2.3.0 release
>> 
>> (Jerry) UIMA-1153 
>> thread safety issue with sample flow controller
>> AdvancedFixedFlowController
>> 
>> (Eddie)UIMA-1038
>> setUimaClasspath.sh
>> should export library paths for interoperability with UIMA C++
>> 
>> (Jerry) UIMA-1531
>> Need a script to launch
>> a UIMA-AS service via RunWithJarPath
>> 
>> (Jerry) UIMA-1484 
>> Update UIMA-AS Documentation
>> 
>>
>> Could the people in (parenthesis) take the lead in looking at these and
>> try and close out these issues?
>>
>> Thanks! -Marshall
>>
> 

Re: Issues not closed for 2.3.0 release

[Jaroslaw Cwiklik]

Marshall, I cant close https://issues.apache.org/jira/browse/UIMA-1531 yet.
Over the weekend I've discovered why the Logger ignores uima
formatter defined in  Logger.properties and logs entries using the default
formatter (xml). It turns out that the java LogManager uses system
classloader to load the formatter class. Here is the method where this magic
occurs:
Formatter getFormatterProperty(String name, Formatter defaultValue) {
String val = getProperty(name);
try {
if (val != null) {
Class clz = ClassLoader.getSystemClassLoader().loadClass(val);
return (Formatter) clz.newInstance();
}
} catch (Exception ex) {
// We got one of a variety of exceptions in creating the
// class or creating an instance.
// Drop through.
}
// We got an exception. Return the defaultValue.
return defaultValue;
}

It surprising that that this java component silently ignores errors. Since
the uima formatter class is not loaded into the system classloader by the
UimaBootstrap, the LogManager returns a default formatter from the method
above.

I dont know how to fix this problem yet. One possibility is to change
UimaBootstrap to load jars/classes into the system classloader. Another
possibility is to load this one class into the system classloader although
I've failed trying to do this. I attempted to augment
the system classloader using reflection to inject a new URL like so:

  URL url = new
URL("jar:file:/C:/uima/releases/2.3.3-04/apache-uima/lib/uima-core.jar!/java/util/logging/XMLFormatter.class");

Class[] parameters = new Class[]{URL.class};
URLClassLoader sysloader = (URLClassLoader)
ClassLoader.getSystemClassLoader();
Class sysclass = URLClassLoader.class;
try {
   Method method = sysclass.getDeclaredMethod("addURL", parameters);
   method.setAccessible(true);
   method.invoke(sysloader, new Object[]{url});
} catch (Throwable t) {
   t.printStackTrace();
   throw new IOException("Error, could not add URL to system
classloader");
}//end try catch

Again, for some reason this doesnt work.I can inject jar files into the
system classloader using the above technique but not individual classes from
jars.


Jerry


On Sun, Oct 18, 2009 at 4:22 PM, Marshall Schor  wrote:

> Hi everyone,
>
> I'm "itching" to get the 2.3.0 release done.
>
> Of the issues that are open, reopened, or in-progress, most are now
> being deferred to past this release.
>
> The ones left that may need to be worked on before the release is done
> include:
>
> (Joern) UIMA-1619 
> Clarify that a Cas Editor Project is required to use the Cas Editor in
> the documentation 
> (Eddie)UIMA-1611 
> UimacppServiceControlled need to implement isStopped()
> 
> (Marshall) UIMA-1539 
> Update READMEs for 2.3.0 release
> 
> (Jerry) UIMA-1153 
> thread safety issue with sample flow controller
> AdvancedFixedFlowController
> 
> (Eddie)UIMA-1038
> setUimaClasspath.sh
> should export library paths for interoperability with UIMA C++
> 
> (Jerry) UIMA-1531
> Need a script to launch
> a UIMA-AS service via RunWithJarPath
> 
> (Jerry) UIMA-1484 
> Update UIMA-AS Documentation
> 
>
> Could the people in (parenthesis) take the lead in looking at these and
> try and close out these issues?
>
> Thanks! -Marshall
>

Re: Issues not closed for 2.3.0 release

[Burn Lewis]

Also I think we need to fix 1607 - wrong log format - Jerry is working on
it.
- Burn.

Re: Issues not closed for 2.3.0 release

[Jörn Kottmann]

Marshall Schor wrote:

Jörn Kottmann wrote:
  

On Oct 18, 2009, at 10:22 PM, Marshall Schor wrote:



(Joern) UIMA-1619 
Clarify that a Cas Editor Project is required to use the Cas Editor in
the documentation 
  

I will close this issue next week,
Jörn


Danke!  I can also do the first pass at this, and then you could just
"check" it -  if it's ok with you?
  

Yes that would be nice, thanks. Otherwise if you don't want to
do it, I think I will have it done until Wednesday.

Jörn

Re: Issues not closed for 2.3.0 release

[Marshall Schor]

Jörn Kottmann wrote:
>
> On Oct 18, 2009, at 10:22 PM, Marshall Schor wrote:
>
>> (Joern) UIMA-1619 
>> Clarify that a Cas Editor Project is required to use the Cas Editor in
>> the documentation 
>
>
> I will close this issue next week,
> Jörn
Danke!  I can also do the first pass at this, and then you could just
"check" it -  if it's ok with you?

-Marshall

Re: Issues not closed for 2.3.0 release

[Jörn Kottmann]

On Oct 18, 2009, at 10:22 PM, Marshall Schor wrote:

(Joern) UIMA-1619 

Clarify that a Cas Editor Project is required to use the Cas Editor in
the documentation 



I will close this issue next week,
Jörn