Jigsaw EA feedback for elasticsearch

2015-09-10 Thread Robert Muir
I just got elasticsearch working on the jigsaw EA build (thanks again,
this is really helpful).

Most fixes were very minor, but #5 and #6 are actual loss of
features/functionality to users.

1. compilation issues I encountered are not related to jigsaw, but
happen for all 9-ea builds. Looks like there is a maven build issue
for 9-ea if you try to use -Werror (and we want to fail on compiler
warnings). We just haven't noticed because of stability issues, they
have not been running in jenkins.
2. we have a "jar hell detector" that threw an
UnsupportedOperationException, because classloader is no longer a
URLClassLoader, so we can't get the list of urls. This caused all
tests to fail. I changed the code to parse java.class.path.
3. we have a "jvm info" api that provides information about the jvm,
e.g. to assist our engineers in debugging different nodes in the
cluster. it was not prepared to handle UnsupportedOperationException
from RuntimeMXBean.getBootClassPath: I fixed it to fall back to
sun.boot.class.path, otherwise fall back to "unknown".
4. exception serialization tests failed, because we manually serialize
exceptions. We previously used java serialization, but it causes
serious trouble because of backwards compatibility breaks between even
minor jdk versions: this would strike when users try to upgrade their
jvms for nodes in their cluster with a rolling restart. The tests fail
because the stacktrace "loses" stuff after deserialization (the module
version). For now i just disabled the tests on java 9, because I don't
know how we can support e.g. java 8 and java 9 and populate this stuff
"optionally" yet without more digging.
5. we have monitoring apis that provide basic system information,
similar to #3, for debugging purposes, and to feed monitoring tools so
people can track the health of the cluster. previously, we used the
sigar library (JNI) for this, but it has bugs that caused users
crashes. So we were forced to limit ourselves to what is provided with
java management apis: which is much less, but we figure it has the
basics. For some very basic stats, this means we also look for
com.sun.management apis
(https://docs.oracle.com/javase/7/docs/jre/api/management/extension/com/sun/management/package-summary.html)
and if they are available, we provide the stuff available there too,
like how much ram is on the machine, swap in use, number of open/max
file descriptors, and so on. We test what is available and what is not
based on platform so we can detect if something changes in the JDK,
like what happens with jigsaw, where they all become unavailable.
These stats are important for debugging: e.g. if lucene is not
behaving right, merging could fall behind and many open
filedescriptors is a sign of that. I disabled these tests for java 9
but I think this is too much, we will be forced to write our own JNA
or /proc reading code for platforms we care about if this is not
available from the JDK, its too important. I'd really really really
like to not have to do that. it seems a little odd to block these apis
but allow Unsafe IMO.
6. cluster snapshot/restore to amazon s3 does not work, because of
their use of internal ssl libraries. I've tried to get them to fix it
for a while now (https://github.com/aws/aws-sdk-java/pull/432). This
is also a serious loss of functionality, if they wont fix it, I guess
we have to fork the aws sdk.
7. we had some test bugs similar to the lucene case (static leaks in
tests and the same test framework tried to compute the size of some
internal class).
8. during testing I hit some kind of bug, where the thai break
iterator returned wrong information. This might be hotspot-related or
something else, and it never reproduced again. We use this check
(https://github.com/apache/lucene-solr/blob/trunk/lucene/analysis/common/src/java/org/apache/lucene/analysis/th/ThaiTokenizer.java#L37-L47)
to see if we can "really" tokenize thai, otherwise we throw an
exception. For some IBM JVM versions at least in the past, they did
not have a breakiterator for thai. I guess it just goes to show the EA
build is really a prototype, and not yet ready to be added to our CI
servers and so on... which is the only way I can ensure this huge
codebase stays working with jigsaw.


RE: Groovy with Jigsaw

2015-09-10 Thread Robert Muir
I agree with Uwe. Practically this just makes more code to fix. I dont see
any advantage of using a new exception, what is the rationale? Using an
existing exception means existing try catch blocks will take care in many
of these cases and just slow the adoption of java 9 even more.

On Sep 10, 2015 6:58 PM, "Uwe Schindler"  wrote:
>
> Hi,
>
> > Now a more practical one... In Groovy we have code like this:
> >
> >  try {
> >  AccessController.doPrivileged(new PrivilegedAction() {
> >  public Object run() {
> >  c.setAccessible(true);
> >  return null;
> >  }
> >  });
> >  }
> >  catch (SecurityException e) {
> >  // IGNORE
> >  }
> >
> > which is code supposed to "check" if we can access the class member or
if
> > any security manager will disallow this. In this case the method becomes
> > unavailable for invocation... Now jigsaw changes do make the
setAccessible
> > method also throws java.lang.reflect.InaccessibleObjectException. What
is
> > the suggested way to handle this for code that has to be also
compatible to
> > pre JDK9 code?
> > catch RuntimeException, check the class name and rethrow if the name
does
> > not fit?
>
> I already opened an issue about this:
https://issues.apache.org/jira/browse/GROOVY-7587
>
> This makes Groovy unuseable at the moment with JIGSAW. Elasticsearch is
affected by this, but also the Ant-based build scripts used to build Lucene
and Forbiddenapis (both use Groovy from Ant). I think the only "quick" fix
would be to check RuntimeException, check class type, otherwise rethrow.
>
> I also wanted to ask the question: setAccessible() already throws some
checked exceptions, why invent a new one that’s no longer checked? Could it
not simply be one already there - one of the checked ones - or a new one
just subclassing ReflectiveOperationException? This is completely
inconsistent to other core reflection APIs. Why have a totally unexpected
new one?
>
> Uwe
>
> -
> Uwe Schindler
> uschind...@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
>


Re: jdeps and annotations parameters

2015-09-10 Thread Jonathan Gibbons



On 09/08/2015 06:12 PM, Alex Buckley wrote:

Jeff,

I confirm (JDK 8u40) that in an instance/static initializer, where a 
local variable declaration includes an annotated type, there is no 
RuntimeVisibleTypeAnnotations attribute emitted for the type 
annotation! This is so even when the correct meta-annotations appear 
on the declaration of the annotation type:


  @Target(ElementType.TYPE_USE)
  @Retention(RetentionPolicy.RUNTIME)

In a ctor, the same local variable declaration does cause a 
RuntimeVisibleTypeAnnotations attribute to be emitted for the type 
annotation.


I assume the difference stems from the fact that an annotation on a 
local variable declaration (the declaration, not its type) was 
traditionally never emitted into the class file, regardless of 
@Target. In SE 8, an annotation on the type of a local variable 
declaration can be emitted into the class file, but presumably some 
old code in javac is special-casing initializers and wasn't updated to 
understand type annotations therein.


Jon,

Can you confirm my presumption?


I could believe there is a bug in this area, if that is what you are asking.

-- Jon



Alex

On 9/8/2015 3:28 PM, Jeff Hain wrote:


Hi Alex.



 >Either this is a huge bug in javac or you're not looking in the right
 >place. The code in an instance initializer compiles into the same 

 >method as the code in a constructor, so I would expect the  
method

 >to have a RuntimeVisibleTypeAnnotations attribute whether the local
 >variable (whose type is annotated) exists in {...} or a ctor.



Here are the classes I parsed to test that (using the same annotation as
I described),
and bellow their respective constant pool, which are quite different,
with the
non-constructor one NOT containing RuntimeVisibleTypeAnnotations
(compiled with JDK8_60, -source 1.8 -target 1.8 (I guess the defaults
anyway)):



/**
  *  method.
  */
public static class MyMethodCodeInitBlock {
 {
 // String will not make it into the class file.

@A_TYPE_USE_R_COMPLEX(nbrClsArr={Long.class},tooStrong="",rounding=RoundingMode.UNNECESSARY,doc=@Documented) 


 Integer foo = 1;
 try {
 MyBlackHole.blackHole(foo);
 } catch (RuntimeException e) {
 throw e;
 }
 }
}

/**
  *  method.
  */
public static class MyMethodCodeConstructor {
 public MyMethodCodeConstructor() {
 // String will not make it into the class file.

@A_TYPE_USE_R_COMPLEX(nbrClsArr={Long.class},tooStrong="",rounding=RoundingMode.UNNECESSARY,doc=@Documented) 


 Integer foo = 1;
 try {
 MyBlackHole.blackHole(foo);
 } catch (RuntimeException e) {
 throw e;
 }
 }
}



pool[4] : class name utf8 index = 22
pool[5] : class name utf8 index = 24
pool[6] : class name utf8 index = 27
pool[7] : utf8 = 
pool[8] : utf8 = ()V
pool[9] : utf8 = Code
pool[10] : utf8 = LineNumberTable
pool[11] : utf8 = StackMapTable
pool[12] : class name utf8 index = 24
pool[13] : class name utf8 index = 28
pool[14] : class name utf8 index = 22
pool[15] : utf8 = SourceFile
pool[16] : utf8 = ClassDepsParserTest.java
pool[17] : descriptor (TAN) utf8 index = 8 = ()V
pool[18] : class name utf8 index = 28
pool[19] : descriptor (TAN) utf8 index = 30 = null
pool[20] : class name utf8 index = 31
pool[21] : descriptor (TAN) utf8 index = 34 = null
pool[22] : utf8 = java/lang/RuntimeException
pool[23] : class name utf8 index = 35
pool[24] : utf8 = 
net/jadecy/code/ClassDepsParserTest$MyMethodCodeInitBlock

pool[25] : utf8 = MyMethodCodeInitBlock
pool[26] : utf8 = InnerClasses
pool[27] : utf8 = java/lang/Object
pool[28] : utf8 = java/lang/Integer
pool[29] : utf8 = valueOf
pool[30] : utf8 = (I)Ljava/lang/Integer;
pool[31] : utf8 = net/jadecy/code/ClassDepsParserTest$MyBlackHole
pool[32] : utf8 = MyBlackHole
pool[33] : utf8 = blackHole
pool[34] : utf8 = (Ljava/lang/Object;)Z
pool[35] : utf8 = net/jadecy/code/ClassDepsParserTest

pool[4] : class name utf8 index = 36
pool[5] : class name utf8 index = 38
pool[6] : class name utf8 index = 40
pool[7] : utf8 = 
pool[8] : utf8 = ()V
pool[9] : utf8 = Code
pool[10] : utf8 = LineNumberTable
pool[11] : utf8 = StackMapTable
pool[12] : class name utf8 index = 38
pool[13] : class name utf8 index = 41
pool[14] : class name utf8 index = 36
pool[15] : utf8 = RuntimeVisibleTypeAnnotations
pool[16] : class name utf8 index = 42
pool[17] : utf8 = A_TYPE_USE_R_COMPLEX
pool[18] : utf8 = InnerClasses
pool[19] : utf8 = 
Lnet/jadecy/code/ClassDepsParserTest$A_TYPE_USE_R_COMPLEX;

pool[20] : utf8 = nbrClsArr
pool[21] : utf8 = Ljava/lang/Long;
pool[22] : utf8 = tooStrong
pool[23] : utf8 =
pool[24] : utf8 = rounding
pool[25] : utf8 = Ljava/math/RoundingMode;
pool[26] : utf8 = UNNECESSARY
pool[27] : utf8 = doc
pool[28] : utf8 = Ljava/lang/annotation/Documented;
pool[29] : utf8 = SourceFile
pool[30] : utf8 = ClassDepsParserTest.java
pool[31] : descriptor (TAN) utf8 index = 8 = ()V
pool[32] : class name utf8 index = 41
p

RE: Groovy with Jigsaw

2015-09-10 Thread Uwe Schindler
Hi,

> Now a more practical one... In Groovy we have code like this:
> 
>  try {
>  AccessController.doPrivileged(new PrivilegedAction() {
>  public Object run() {
>  c.setAccessible(true);
>  return null;
>  }
>  });
>  }
>  catch (SecurityException e) {
>  // IGNORE
>  }
> 
> which is code supposed to "check" if we can access the class member or if
> any security manager will disallow this. In this case the method becomes
> unavailable for invocation... Now jigsaw changes do make the setAccessible
> method also throws java.lang.reflect.InaccessibleObjectException. What is
> the suggested way to handle this for code that has to be also compatible to
> pre JDK9 code?
> catch RuntimeException, check the class name and rethrow if the name does
> not fit?

I already opened an issue about this: 
https://issues.apache.org/jira/browse/GROOVY-7587

This makes Groovy unuseable at the moment with JIGSAW. Elasticsearch is 
affected by this, but also the Ant-based build scripts used to build Lucene and 
Forbiddenapis (both use Groovy from Ant). I think the only "quick" fix would be 
to check RuntimeException, check class type, otherwise rethrow.

I also wanted to ask the question: setAccessible() already throws some checked 
exceptions, why invent a new one that’s no longer checked? Could it not simply 
be one already there - one of the checked ones - or a new one just subclassing 
ReflectiveOperationException? This is completely inconsistent to other core 
reflection APIs. Why have a totally unexpected new one?

Uwe

-
Uwe Schindler
uschind...@apache.org 
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/




Groovy with Jigsaw

2015-09-10 Thread Jochen Theodorou

Hi,

we are currently investigating problems Groovy has with Jigsaw. A lot 
changed with jigsaw, and since Groovy does often not use standard 
methods, a lot fails.


But first I would like to ask something that I haven't seen covered in 
http://openjdk.java.net/projects/jigsaw/quick-start or 
http://openjdk.java.net/projects/jigsaw/spec/sotms/ Maybe I did not see 
it, since it is late here. I hope the questions should not better have 
gone to the expert group comment list. It was not fully clear to me 
where goes what. Anyway...


As far as I understood a classloader can have a N modules plus the 
unnamed module. I assume a the children of a classloader do not 
automatically share modules (or not at all?) with their children.

Do assume right in that there is no way to create a module at runtime?

Next question is about runtime generated classes. And I mean here 
classes generated by user code at runtime, as well as proxies, code for 
reflection and invokedynamic and all the other bytecode generating 
facilities. Will they all go into the unnamed module?


Assuming they go there (how else can you choose from those N modules), 
what access rights to the modules will they have? I assume to everything 
that is exported, and for that it does not matter if it is the same 
class loader or a parent or any other loader.


I further assume you can load a module at runtime... by for example 
using URLClassLoader. Now let us say there is a classloader for module 
A, which exports some packages to module B. Must module B then be loaded 
in the same classloader (or a parent) as A? If a child loader can load B 
and allow access according to the exports rules. What happens if 
multiple children each have their own B, and maybe those B are totally 
unrelated to each other? I assume this is possible.


Now a more practical one... In Groovy we have code like this:

try {
AccessController.doPrivileged(new PrivilegedAction() {
public Object run() {
c.setAccessible(true);
return null;
}
});
}
catch (SecurityException e) {
// IGNORE
}

which is code supposed to "check" if we can access the class member or 
if any security manager will disallow this. In this case the method 
becomes unavailable for invocation... Now jigsaw changes do make the 
setAccessible method also throws 
java.lang.reflect.InaccessibleObjectException. What is the suggested way 
to handle this for code that has to be also compatible to pre JDK9 code? 
catch RuntimeException, check the class name and rethrow if the name 
does not fit?


And then a question probably a bit more complicated...

Groovy uses reflection/runtime generated classes/invokedynamic to invoke 
methods. That means the method invocation point (for reflection, the 
class that calls the invoke method) is most often in the Groovy runtime. 
And even if Groovy would define its own module, those runtime generated 
classes would not be in the same module (or is there a way for this?) 
Now assume somebody wrote this class in Groovy:


/* code without sense following */
class MyFactory {
  public Factory getFactory() {return new FactoryImpl()}
}
interface Factory{String foo()}
class FactoryImpl implements Factory{
  String foo() { return internal()+"foo" }
  String internal() {"Factory"}
}

and let us assume MyFactory and Factory are in a package, which is 
exported, FactoryImpl not. Now... since Groovy is currently in no module 
and since it needs to use reflection to execute the internal() method 
call here, does this mean the call will fail, because the object is 
inaccessible? Actually I assume we would not even see the method.


If that assumption is right, does this means every application that uses 
modules and is at least partially written in Groovy, has to export each 
and every package to Groovy, in order to allow it to do method calls? 
That is assuming Groovy has a module by then. Without it I assume it 
cannot be made.


Well... enough for now, further questions will surely develop later on.

bye Jochen Theodorou

--
Jochen "blackdrag" Theodorou
blog: http://blackdragsview.blogspot.com/



Re: Initial module-system design, API, and early-access builds

2015-09-10 Thread mark . reinhold
2015/9/10 3:00 -0700, snfu...@gmx.de:
> reading the overview document I stumbled upon the sentence, that the 
> module declaration should by convention go to a file named 
> module-info.java and therefore complied to ||module-info.class file in 
> the root directory of the jar file. Does this mean, that a modular jar 
> file may only contain one module at a time?

Yes.

> For many applications it is 
> common practice to fold all compiled classes and its dependencies into 
> one big jar file. Would this still work?
> Wouldn't it be better to include the module name in the file name?
> e.g.: com-foo-bar-module-info.java

The assumption behind the modular-JAR concept is that a JAR file is a
single unit of release and re-use, rather than a collection of all the
components that make up an entire application, so there's no need to
package more than one module into such a JAR.

If you think it's critical for modular JARs to support multiple modules
then please let the EG know (jpms-spec-comme...@openjdk.java.net).

- Mark


Re: Project Jigsaw: Early-Access Builds available on jdk9.java.net/jigsaw

2015-09-10 Thread Mandy Chung



On 09/10/2015 01:55 PM, Jim Connors wrote:


When you run the com.greetings.Main class via:

$ java -mp mods -m com.greetings/com.greetings.Main

The Quick Start guide shows the output as:

class org.fastsocket.FastNetworkSocket

The Main.java file referenced above prints nothing (at least in my 
case), would it have to be modified slightly, something like:


$ cat src/com.greetings/com/greetings/Main.java
package com.greetings;

import com.socket.NetworkSocket;

public class Main {
public static void main(String[] args) {
NetworkSocket s = NetworkSocket.open();
System.out.println(s.getClass().toString());
   }
}



I think you are right.  The System.out.println is missing in the sample 
code in the quickstart guide.  The key point there is to show that the 
service provider implementation class is loaded.


Mandy

Mandy


Re: Initial module-system design, API, and early-access builds

2015-09-10 Thread Stefan Fuchs

Hi,

reading the overview document I stumbled upon the sentence, that the 
module declaration should by convention go to a file named 
module-info.java and therefore complied to ||module-info.class file in 
the root directory of the jar file. Does this mean, that a modular jar 
file may only contain one module at a time? For many applications it is 
common practice to fold all compiled classes and its dependencies into 
one big jar file. Would this still work?

Wouldn't it be better to include the module name in the file name?
e.g.: com-foo-bar-module-info.java

- Stefan




Re: Another Quick Start Guide Nitpick

2015-09-10 Thread Mani Sarkar
Jim,

Yes that's another issue there. I meant to report - already switched it
around in my bash scripts see the 5_Missing_xxx folders.

One more thing while we are at it - wherever possible in the examples we
need to mention that the javac and java commands should be executed from
outside the src folder - its obviously but just to be a bit more clear, as
these are new things we are learning about JDK9.

Cheers,
Mani

On Thu, Sep 10, 2015 at 10:24 PM, Jim Connors 
wrote:

> One more thing while I'm at it:
>
> In the "Missing requires or missing exports" section:
> http://openjdk.java.net/projects/jigsaw/quick-start#missing
>
> We now fix this module declaration but introduce a different mistake, this
> time we omit the exports from the org.astro module declaration:
>
> $ cat src/com.greetings/module-info.java
> module com.greetings {
> requires org.astro;
> }
> $ cat src/org.astro/module-info.java
> module org.astro {
> // exports org.astro;
> }
>
>
> Prior to issuing the javac on the greetings.com module (with anticipated
> error)
>
> $ javac -modulepath mods -d mods/com.greetings \
>src/com.greetings/module-info.java 
> src/com.greetings/com/greetings/Main.java
>
>
> I think the org.astro module has to be rebuilt first.  Otherwise no error
> will appear.
>
> $ javac -d mods/org.astro \
> src/org.astro/module-info.java src/org.astro/org/astro/World.java
>
> Make sense?
> -- Jim C
>



-- 
@theNeomatrix369 *  |  **Blog
**  |  *LJC Associate & LJC Advocate
(@adoptopenjdk & @adoptajsr programs)
*Meet-a-Project - *MutabilityDetector
*  |  **Bitbucket
* * |  **Github
* * |  **LinkedIn
*
*Come to Devoxx UK 2016:* http://www.devoxx.co.uk/

*Don't chase success, rather aim for "Excellence", and success will come
chasing after you!*


Another Quick Start Guide Nitpick

2015-09-10 Thread Jim Connors

One more thing while I'm at it:

In the "Missing requires or missing exports" section:
http://openjdk.java.net/projects/jigsaw/quick-start#missing

   We now fix this module declaration but introduce a different
   mistake, this time we omit the |exports| from the |org.astro| module
   declaration:

$ cat src/com.greetings/module-info.java
module com.greetings {
requires org.astro;
}
$ cat src/org.astro/module-info.java
module org.astro {
// exports org.astro;
}


Prior to issuing the javac on the greetings.com module (with anticipated 
error)


$ javac -modulepath mods -d mods/com.greetings \
   src/com.greetings/module-info.java 
src/com.greetings/com/greetings/Main.java


I think the org.astro module has to be rebuilt first.  Otherwise no 
error will appear.


$ javac -d mods/org.astro \
src/org.astro/module-info.java src/org.astro/org/astro/World.java

Make sense?
-- Jim C


Re: Project Jigsaw: Early-Access Builds available on jdk9.java.net/jigsaw

2015-09-10 Thread Mani Sarkar
Hi Alan,

Thanks for the detailed explanation and it would be great to find this in
your docs - I will link to it.

The reasons I understood, jimage might go away, as I read this https://bugs.
*openjdk*.java.net/browse/JDK-8049369.

But now its clear, they both have their place - I did think of jimage as
the runtime container - thanks for confirming

Cheers,
Mani


On Thu, Sep 10, 2015 at 9:52 PM, Jim Connors 
wrote:

> On 9/10/2015 4:30 PM, Alan Bateman wrote:
>
> I don't immediately see the error in the services example (I need to look
> closer) but I think you have a point that this example jumps ahead too
> much. Good idea to get the examples into a repo with scripts.
>
>
> In the Services Section:
> http://openjdk.java.net/projects/jigsaw/quick-start#services
>
> $ cat src/com.greetings/com/greetings/Main.java
> package com.greetings;
>
> import com.socket.NetworkSocket;
>
> public class Main {
> public static void main(String[] args) {
> NetworkSocket s = NetworkSocket.open();
> }
> }
>
> When you run the com.greetings.Main class via:
>
> $ java -mp mods -m com.greetings/com.greetings.Main
>
> The Quick Start guide shows the output as:
>
> class org.fastsocket.FastNetworkSocket
>
> The Main.java file referenced above prints nothing (at least in my case),
> would it have to be modified slightly, something like:
>
> $ cat src/com.greetings/com/greetings/Main.java
> package com.greetings;
>
> import com.socket.NetworkSocket;
>
> public class Main {
> public static void main(String[] args) {
> NetworkSocket s = NetworkSocket.open();
> System.out.println(s.getClass().toString());
>}
> }
>
> -- Jim C
>



-- 
@theNeomatrix369 *  |  **Blog
**  |  *LJC Associate & LJC Advocate
(@adoptopenjdk & @adoptajsr programs)
*Meet-a-Project - *MutabilityDetector
*  |  **Bitbucket
* * |  **Github
* * |  **LinkedIn
*
*Come to Devoxx UK 2016:* http://www.devoxx.co.uk/

*Don't chase success, rather aim for "Excellence", and success will come
chasing after you!*


Re: Project Jigsaw: Early-Access Builds available on jdk9.java.net/jigsaw

2015-09-10 Thread Mani Sarkar
Jim you stole my thunder and lightening - that was it.

Cheers,
Mani

On Thu, Sep 10, 2015 at 9:55 PM, Jim Connors 
wrote:

> On 9/10/2015 4:30 PM, Alan Bateman wrote:
>
> I don't immediately see the error in the services example (I need to look
> closer) but I think you have a point that this example jumps ahead too
> much. Good idea to get the examples into a repo with scripts.
>
>
> In the Services Section:
> http://openjdk.java.net/projects/jigsaw/quick-start#services
>
> $ cat src/com.greetings/com/greetings/Main.java
> package com.greetings;
>
> import com.socket.NetworkSocket;
>
> public class Main {
> public static void main(String[] args) {
> NetworkSocket s = NetworkSocket.open();
> }
> }
>
> When you run the com.greetings.Main class via:
>
> $ java -mp mods -m com.greetings/com.greetings.Main
>
> The Quick Start guide shows the output as:
>
> class org.fastsocket.FastNetworkSocket
>
> The Main.java file referenced above prints nothing (at least in my case),
> would it have to be modified slightly, something like:
>
> $ cat src/com.greetings/com/greetings/Main.java
> package com.greetings;
>
> import com.socket.NetworkSocket;
>
> public class Main {
> public static void main(String[] args) {
> NetworkSocket s = NetworkSocket.open();
> System.out.println(s.getClass().toString());
>}
> }
>
> -- Jim C
>



-- 
@theNeomatrix369 *  |  **Blog
**  |  *LJC Associate & LJC Advocate
(@adoptopenjdk & @adoptajsr programs)
*Meet-a-Project - *MutabilityDetector
*  |  **Bitbucket
* * |  **Github
* * |  **LinkedIn
*
*Come to Devoxx UK 2016:* http://www.devoxx.co.uk/

*Don't chase success, rather aim for "Excellence", and success will come
chasing after you!*


Re: Project Jigsaw: Early-Access Builds available on jdk9.java.net/jigsaw

2015-09-10 Thread Jim Connors

On 9/10/2015 4:30 PM, Alan Bateman wrote:
I don't immediately see the error in the services example (I need to 
look closer) but I think you have a point that this example jumps 
ahead too much. Good idea to get the examples into a repo with scripts.




In the Services Section: 
http://openjdk.java.net/projects/jigsaw/quick-start#services


$ cat src/com.greetings/com/greetings/Main.java
package com.greetings;

import com.socket.NetworkSocket;

public class Main {
public static void main(String[] args) {
NetworkSocket s = NetworkSocket.open();
}
}

When you run the com.greetings.Main class via:

$ java -mp mods -m com.greetings/com.greetings.Main

The Quick Start guide shows the output as:

class org.fastsocket.FastNetworkSocket

The Main.java file referenced above prints nothing (at least in my 
case), would it have to be modified slightly, something like:


$ cat src/com.greetings/com/greetings/Main.java
package com.greetings;

import com.socket.NetworkSocket;

public class Main {
public static void main(String[] args) {
NetworkSocket s = NetworkSocket.open();
System.out.println(s.getClass().toString());
   }
}

-- Jim C


Re: Project Jigsaw: Early-Access Builds available on jdk9.java.net/jigsaw

2015-09-10 Thread Jim Connors

On 9/10/2015 4:30 PM, Alan Bateman wrote:
I don't immediately see the error in the services example (I need to 
look closer) but I think you have a point that this example jumps 
ahead too much. Good idea to get the examples into a repo with scripts.




In the Services Section: 
http://openjdk.java.net/projects/jigsaw/quick-start#services


$ cat src/com.greetings/com/greetings/Main.java
package com.greetings;

import com.socket.NetworkSocket;

public class Main {
public static void main(String[] args) {
NetworkSocket s = NetworkSocket.open();
}
}

When you run the com.greetings.Main class via:

$ java -mp mods -m com.greetings/com.greetings.Main

The Quick Start guide shows the output as:

class org.fastsocket.FastNetworkSocket

The Main.java file referenced above prints nothing (at least in my 
case), would it have to be modified slightly, something like:


$ cat src/com.greetings/com/greetings/Main.java
package com.greetings;

import com.socket.NetworkSocket;

public class Main {
public static void main(String[] args) {
NetworkSocket s = NetworkSocket.open();
System.out.println(s.getClass().toString());
   }
}

-- Jim C


Re: Project Jigsaw: Early-Access Builds available on jdk9.java.net/jigsaw

2015-09-10 Thread Alan Bateman

On 10/09/2015 17:15, Mani Sarkar wrote:

Hi Rory,

I have played with the Jigsaw JDK using the Quick starter guide and 
have this to share with our community:

https://github.com/neomatrix369/jdk9-jigsaw

There is a small mistake in the example on Services (see my 
implementation you will spot it soon), also the Services example 
explains more than one concept at a time and I think this might just 
be a bit more than a simple example to illustrate Jigsaw features.


The last two examples are not fully fleshed out. I would put a 
footnote about jlink, that jmod + tools are going to supersede jimage 
(if from what I have read so far is correct, I have even seen this 
mentioned on a JBUG issue).


I don't immediately see the error in the services example (I need to 
look closer) but I think you have a point that this example jumps ahead 
too much. Good idea to get the examples into a repo with scripts.


Just on the tools then jlink + jmod don't supersede the jimage tool. 
Lots of new j- tools so the following might be helpful:


jlink is the linker. It links a set of modules together to create a 
run-time image, the layout of which is described in JEP 220. The jlink 
tool is in the current EA builds but no documentation to point at yet. 
There will be a JEP in time (as mentioned already in JEP 261). The jlink 
example you see in the quick start page should work and create a 
run-time image that contains module com.greetings and its transitive 
dependences. Another thing to mention is that the jigsaw/jake builds 
invoke jlink to create the JDK, JRE, and compact profile images. I think 
of jlink as a tool for the advanced tool chain and not something that 
most developers will use directly.


The jmod tool is to the JMOD format as the jar tool is to the JAR 
format. In the jigsaw/jake build then you'll see that the tool is used 
to create a JMOD for each standard and JDK-specific module. The 
directory of JMOD files then serves as the module path for the linker 
when it creates the JDK and JRE images. Like jlink then I don't expect 
too many developers to need to use the jmod tool or JMOD format as 
modular JAR files will do just fine for most modules. In the example in 
the quick start page then the only reason that the JDK jmods directory 
is on the module path is because creating a custom image requires 
resolving standard and maybe JDK modules so the tool needs to be able to 
locate the corresponding module artifacts.


Now jimage. In a run-time image (JEP 220) then classes and resources are 
stored in a container that is in jimage format. The format is 
deliberated not documented and best to assume it will change and evolve 
over time as it inhales new optimizations. Introducing a new container 
format is a bit scary and needs some tooling to aid troubleshooting in 
the event of problems. This is why the JDK 9 builds and the EA builds 
have a jimage tool. It can be used to do some verification, list or 
extract the contents. It's not meant to be on par with the jar tool of 
course. Instead just think of the jimage tool as First Aid Kit hanging 
on the wall in the event that you might need it.


-Alan.




hg: jigsaw/jake/hotspot: Move code out of locked regsion to aid moving of logging code

2015-09-10 Thread alan . bateman
Changeset: 46f34d9a3520
Author:hseigel
Date:  2015-09-10 20:42 +0100
URL:   http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/46f34d9a3520

Move code out of locked regsion to aid moving of logging code

! src/share/vm/classfile/modules.cpp



hg: jigsaw/jake/jdk: javadoc comment needs to be before package declaration

2015-09-10 Thread alan . bateman
Changeset: 80c9be2cf90b
Author:alanb
Date:  2015-09-10 15:48 +0100
URL:   http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/80c9be2cf90b

javadoc comment needs to be before package declaration

! src/java.base/share/classes/java/lang/module/package-info.java



Re: jigsaw EA feedback for apache lucene

2015-09-10 Thread Robert Muir
On Thu, Sep 10, 2015 at 8:10 AM, Alan Bateman  wrote:

>
> If I understand you then you are finding these usages with JDK 7 and JDK 8
> because you're forced to configure the security policy to grant access to
> these sun.* packages. So some extent then it means you have a head start.
>

Yes, thats right. We use the "forbidden-apis" tool Uwe mentions to
statically scan for these classes, it will fail the build on usage of
internal java apis, but it doesn't detect any accesses by reflection
and isn't yet easy to apply to all your third party dependencies, too
(https://github.com/policeman-tools/forbidden-apis/issues/38).

We have used securitymanager in tests for some time now, for a number
of reasons (e.g. detect bugs like bad filesystem accesses), and have
been trying to minimize the permissions over time. Of course this only
works when there is proper test coverage, and only recently (last
year) did we become more aggressive and e.g. break out
RuntimePermission("*") as we became more conscious of the problems and
the benefits of cleaning it up.

We also really try to keep dependencies minimal for both software projects.

So I guess thats just more explanation of our experience/concerns.
Personally I really really like the idea here, but I am a bit worried
about projects especially with many dependencies that don't have this
tooling etc in place (and ours is far from perfect), and might grossly
underestimate the amount of effort needed to get things working. I
don't speak of just internal package accesses, but also stuff like
this, look at the top voted answer, its not
URLClassLoader.newInstance() or anything like that:

http://stackoverflow.com/questions/60764/how-should-i-load-jars-dynamically-at-runtime

We just fixed that one in elasticsearch only this spring/summer and it
required revamping the entire plugin mechanism from single to
multi-classloader, this was a lot of code to clean up.


Re: jigsaw EA feedback for Eclipse/JBoss Tools and WildFly

2015-09-10 Thread Alan Bateman


On 10/09/2015 14:21, Rory O'Donnell wrote:
Martin - thanks for your feedback, not sure your email made it to the 
list without subscribing.


It will now, rgds Rory
On 10/09/2015 13:59, Martin Malina wrote:


I reported it here: https://issues.jboss.org/browse/JBIDE-20635


Thanks for forwarding. The interesting line seems to be:

13:03:04,020 ERROR [org.jboss.as.controller] (Controller Boot Thread) 
WFLYCTL0002: Error booting the container: java.lang.IllegalAccessError: 
Class __redirected.__XMLInputFactory can not access a member of class 
com.sun.xml.internal.stream.XMLInputFactoryImpl (module java.xml) with 
modifiers "public", module java.xml does not export 
com.sun.xml.internal.stream to 


So __redirected.__XMLInputFactory (WildFly?, generated?) is attempting 
to use Class.newInstance to instantiate 
com.sun.xml.internal.stream.XMLInputFactoryImpl. This should fail with 
IllegalAccessException because com.sun.xml.internal.stream is not 
exported. I can't tell where IllegalAccessError is coming from, maybe 
someone is catching IllegalAccessException and re-throwing it as an 
IllegalAccessError.


I assume using -XaddExports can be used to workaround this temporarily.

-Alan.



RE: jigsaw EA feedback for apache lucene

2015-09-10 Thread Uwe Schindler
Hi Alan,

thanks for the useful information. I was able to fix the forbidden-apis checker 
to work correctly using the Class#getModule() approach. This can be done with 
easy core reflection also from older code that has to compile against Java 6. I 
tested it with the nightly build of b80 and works perfectly, whole testsuite of 
the checker passes.

Here is the description what I do:
https://goo.gl/9oT2Ch

And this is the patch:
https://goo.gl/noOYgD

I still think that the resource encapsulation is a good thing, but on the other 
hand at least the special case "get bytecode of class I have access to" should 
be supported, maybe deprecated.

Uwe

-
Uwe Schindler
uschind...@apache.org 
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/


> -Original Message-
> From: Alan Bateman [mailto:alan.bate...@oracle.com]
> Sent: Thursday, September 10, 2015 2:47 PM
> To: Uwe Schindler; jigsaw-dev
> Subject: Re: jigsaw EA feedback for apache lucene
> 
> Uwe Schindler wrote:
> > So you would first to find out the module of a class and then ask the
> module for the resource. Is there a way to get the module from the binary
> (string a la Class#forName()) class name without actually loading the class?
> Yes, you'd need the Class object so you can call getModule().
> 
> >
> > The second problem for such checkers is: Unfortunately this needs Java 9
> at compile time... On solution for this would be to wrap the checker's 
> internal
> loading mechanism with a class solely compiled for Java 9 at runtime, if Java 
> 9
> was detected - that provides the missing resources - older Java versions
> would use the approach as currently implemented.
> This will work, or would using core reflection. I assume you've seen JEP
> 238 and the proposal for multi-release JAR files.
> 
> 
> > :
> >
> >
> > Is there a reason why the (system-) classloader does not return any
> resources from foreign modules anymore? I agree that some internal
> properties files and so on are not needed to be available, but if a caller 
> can do
> Class#forName() on a specific class that is public on top, why should it not
> also do a getResource() call on the class's bytecode - this makes no sense to
> me? Many frameworks like Google Guice, Spring,... rely on that because they
> may need to create proxy classes and therefore need to have the bytecode.
> So to me this seems like a major break!
> One of the design issues that the JSR will need to decide is whether
> resources in a module are encapsulated, see:
> http://openjdk.java.net/projects/jigsaw/spec/reqs/#resource-
> encapsulation
> 
> As things stand in the current builds then code in a module can get at
> any of the resources in its own module (Class.getResourceAsStream) but
> it can't use the legacy APIs to get at resources in other named modules.
> The legacy APIs work exactly as before for resources in unnamed modules
> (think JARs on the class path). Fameworks that scan the class path for
> resources will work as before but would need changes to get at resources
> in named modules.
> 
> -Alan



Re: jigsaw EA feedback for Eclipse/JBoss Tools and WildFly

2015-09-10 Thread Rory O'Donnell
Martin - thanks for your feedback, not sure your email made it to the 
list without subscribing.


It will now, rgds Rory
On 10/09/2015 13:59, Martin Malina wrote:

Hi,

Today I tried out the early access build of jdk 1.9 with jigsaw [1]. I 
tried it with JBoss Developer Studio 9.0.0.CR1 nightly build [2] 
(based on Eclipse Mars) and while I didn't see any breakage in 
the tooling itself, I was not able to start WildFly 10 Beta2 [3] with 
this from Eclipse. It would start fine from command line.

I reported it here: https://issues.jboss.org/browse/JBIDE-20635

Rory O'Donnell encouraged me to share this with this list, so here you 
go :)


Note: I'm not subscribed to this mailing list, so please keep me in Cc 
if you have any questions.


Thanks,
Martin


[1] http://jdk9.java.net/jigsaw
[2] 
https://devstudio.redhat.com/9.0/snapshots/builds/devstudio.product_master/

[3] http://wildfly.org/downloads/

--
Martin Malina
JBoss QA Engineer
Red Hat Czech s.r.o.
Purkynova 99
612 45 Brno, Czech Republic

Tel.: +420 532 294 265





--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland



Re: jigsaw EA feedback for apache lucene

2015-09-10 Thread Alan Bateman

Uwe Schindler wrote:

So you would first to find out the module of a class and then ask the module 
for the resource. Is there a way to get the module from the binary (string a la 
Class#forName()) class name without actually loading the class?

Yes, you'd need the Class object so you can call getModule().



The second problem for such checkers is: Unfortunately this needs Java 9 at 
compile time... On solution for this would be to wrap the checker's internal 
loading mechanism with a class solely compiled for Java 9 at runtime, if Java 9 
was detected - that provides the missing resources - older Java versions would 
use the approach as currently implemented.
This will work, or would using core reflection. I assume you've seen JEP 
238 and the proposal for multi-release JAR files.




:


Is there a reason why the (system-) classloader does not return any resources 
from foreign modules anymore? I agree that some internal properties files and 
so on are not needed to be available, but if a caller can do Class#forName() on 
a specific class that is public on top, why should it not also do a 
getResource() call on the class's bytecode - this makes no sense to me? Many 
frameworks like Google Guice, Spring,... rely on that because they may need to 
create proxy classes and therefore need to have the bytecode. So to me this 
seems like a major break!
One of the design issues that the JSR will need to decide is whether 
resources in a module are encapsulated, see:

http://openjdk.java.net/projects/jigsaw/spec/reqs/#resource-encapsulation

As things stand in the current builds then code in a module can get at 
any of the resources in its own module (Class.getResourceAsStream) but 
it can't use the legacy APIs to get at resources in other named modules. 
The legacy APIs work exactly as before for resources in unnamed modules 
(think JARs on the class path). Fameworks that scan the class path for 
resources will work as before but would need changes to get at resources 
in named modules.


-Alan


Re: jigsaw EA feedback for apache lucene

2015-09-10 Thread Alan Bateman

On 10/09/2015 12:22, Robert Muir wrote:

:

When looking at the current situation for elasticsearch, as far as
internal package usage, we aren't doing it ourselves, but today i've
still got to worry about:

1. sun.misc.* for the mmap "unmap hack" of lucene.
2. sun.security.ssl.* for some crazy code in the AWS api.
3. sun.reflect.* for something groovy is doing.
Some of these are may be "critical internal APIs" described in JEP 260 
[1] so there is more time to work on those.




Its rare that java libraries use these internal packages in a "nice"
way (e.g. well-contained in accesscontroller block, fallback if its
not allowed/optional,. ...). Usually it is just direct usage and
nobody is aware or cares about it, because most people are not using
securitymanager.
If I understand you then you are finding these usages with JDK 7 and JDK 
8 because you're forced to configure the security policy to grant access 
to these sun.* packages. So some extent then it means you have a head start.


-Alan.

[1] http://openjdk.java.net/jeps/260


Re: jigsaw EA feedback for apache lucene

2015-09-10 Thread Dawid Weiss
> Yes, unfortunately when it comes to java libs this is "widespread" to
> say the least.
>

Often this isn't even intentional. For example the failure in the static
memory leak checker isn't caused by its codebase (no private class is
referenced); the exception is thrown at runtime when test case fields are
collected and there is an attempt to measure the size of leftover objects.
If somebody leaves an instance of Timezone or Locale this leads to a
runtime security exception.

Dawid


Re: jigsaw EA feedback for apache lucene

2015-09-10 Thread Robert Muir
On Thu, Sep 10, 2015 at 4:21 AM, Alan Bateman  wrote:

>
>> * Cleaning up a bunch of bad package accesses, these were already
>> TODOs in our test security policy, mostly just test bugs and the like.
>
> "bad package accesses", do you mean direct use of JDK-internal classes? As
> you mention security policy then I will guess that these must be granting of
> accessClassInPackage.$PKG because the default policy when running with a
> security manager does not allow direct access to sun.* APIs.
>

Yes, unfortunately when it comes to java libs this is "widespread" to
say the least.

When looking at the current situation for elasticsearch, as far as
internal package usage, we aren't doing it ourselves, but today i've
still got to worry about:

1. sun.misc.* for the mmap "unmap hack" of lucene.
2. sun.security.ssl.* for some crazy code in the AWS api.
3. sun.reflect.* for something groovy is doing.

Its rare that java libraries use these internal packages in a "nice"
way (e.g. well-contained in accesscontroller block, fallback if its
not allowed/optional,. ...). Usually it is just direct usage and
nobody is aware or cares about it, because most people are not using
securitymanager.

Third party developers don't seem to care when you throw the
securitymanager argument at them, e.g. i've raised #2 with AWS months
ago and nobody has done anything. So I am happy to see this stuff
enforced more by jigsaw, because it gives me a stronger argument than
my current screaming "this is really screwed up, its a security
problem, etc".

The downside is I do know a lot of code/libraries has issues like
this, because I've been trying to whittle down the security
permissions for this crazy stuff for months. That's why I think it
will impact a lot of developers: I am sure lots of code does stuff
like integrate with AWS, or integrate scripting languages like groovy,
etc. Its harder to fix when its a dependency.


RE: jigsaw EA feedback for apache lucene

2015-09-10 Thread Uwe Schindler
Hi,

> -Original Message-
> From: jigsaw-dev [mailto:jigsaw-dev-boun...@openjdk.java.net] On Behalf
> Of Alan Bateman
> Sent: Thursday, September 10, 2015 12:23 PM
> To: Sundararajan Athijegannathan; Robert Muir
> Cc: jigsaw-dev@openjdk.java.net
> Subject: Re: jigsaw EA feedback for apache lucene
> 
> On 10/09/2015 06:52, Sundararajan Athijegannathan wrote:
> > :
> >
> > ClassLoader.getResource for a .class resource is checked to detect if
> > it runs on jdk9. This returns a "jrt:" URL in jdk9 EA build - but,
> > returns null with jigsaw build. Also,
> > RuntimeMXBean.isBootClassPathSupported() returns false.
> > So, isSupportedJDK will be false.
> >
> > Better solution would be to check if "jrt" nio file system is
> > available & use the same to check and/or read .class files of JDK
> > classes.
> That will work but it might be better to look at
> java.lang.reflect.Module#getResourceAsStream as this will work for
> resources in both named and unnamed modules (for unnamed modules
> then it works like ClassLoader#getResourceAsStream). The summary on this
> is that the ClassLoader#getResourceXXX methods will locate resources on
> the class path as before, it's just that they don't locate resources in named
> modules.

OK. So you would first to find out the module of a class and then ask the 
module for the resource. Is there a way to get the module from the binary 
(string a la Class#forName()) class name without actually loading the class?

The second problem for such checkers is: Unfortunately this needs Java 9 at 
compile time... On solution for this would be to wrap the checker's internal 
loading mechanism with a class solely compiled for Java 9 at runtime, if Java 9 
was detected - that provides the missing resources - older Java versions would 
use the approach as currently implemented.

But the big question is, that I was not able to understand after reading all 
the JIGSAW specs:

Is there a reason why the (system-) classloader does not return any resources 
from foreign modules anymore? I agree that some internal properties files and 
so on are not needed to be available, but if a caller can do Class#forName() on 
a specific class that is public on top, why should it not also do a 
getResource() call on the class's bytecode - this makes no sense to me? Many 
frameworks like Google Guice, Spring,... rely on that because they may need to 
create proxy classes and therefore need to have the bytecode. So to me this 
seems like a major break!

In my opinion, it should be possible to access the bytecode of classes if the 
class itself is available in Class#forName(). This was an assumption you were 
always able to count on. Java 9 does not simplify this, it makes tool building 
more complicated. I was so happy that you added "jrt:" URLs, but this is now 
also gone. The reason why this should work with Classloader is the defined way 
to lookup stuff. If I need a special case to load the bytecode for already 
visible classes, this makes tool development horrible. Since Java 1.0 this was 
a very easy thing. Class#forName() and Class#getResource() used the same lookup 
mechanis hidden by the class loader. You were able to rely since Java 1.0 on 
the fact that if you can load a class with ClassLoader#forName(), you can also 
load its bytecode with the same classloader.

Maybe you should just hide private resources, but if the classloader detects 
that you do a getResource() on resource filename with ".class", it should use 
the same access checks of the module system and return the stuff as stream or 
jrt:// URL. Otherwise return null when Class#forName() would throw 
ClassNotFoundException.

Uwe



Re: jigsaw EA feedback for apache lucene

2015-09-10 Thread Alan Bateman

On 10/09/2015 06:52, Sundararajan Athijegannathan wrote:

:

ClassLoader.getResource for a .class resource is checked to detect if 
it runs on jdk9. This returns a "jrt:" URL in jdk9 EA build - but, 
returns null with jigsaw build. Also, 
RuntimeMXBean.isBootClassPathSupported() returns false.

So, isSupportedJDK will be false.

Better solution would be to check if "jrt" nio file system is 
available & use the same to check and/or read .class files of JDK 
classes.
That will work but it might be better to look at 
java.lang.reflect.Module#getResourceAsStream as this will work for 
resources in both named and unnamed modules (for unnamed modules then it 
works like ClassLoader#getResourceAsStream). The summary on this is that 
the ClassLoader#getResourceXXX methods will locate resources on the 
class path as before, it's just that they don't locate resources in 
named modules.


-Alan


RE: jigsaw EA feedback for apache lucene

2015-09-10 Thread Uwe Schindler
Hi,
 
> On 10/09/2015 06:06, Robert Muir wrote:
> > :
> >
> > Problems were basically all test/build related:
> > * API oddities around FileChannel required a better hack (thanks
> > Brian):
> > http://mail.openjdk.java.net/pipermail/nio-dev/2015-
> September/003322.h
> > tml
> This one is subtle is that it was using core reflection to call a protected
> method in sun.nio.ch.FileChannelImpl. Static analysis with tools such as jdeps
> don't find cases like this. Good to see that a solution was found to this.
> 
> > * Cleaning up a bunch of bad package accesses, these were already
> > TODOs in our test security policy, mostly just test bugs and the like.
> "bad package accesses", do you mean direct use of JDK-internal classes?
> As you mention security policy then I will guess that these must be granting
> of accessClassInPackage.$PKG because the default policy when running with
> a security manager does not allow direct access to sun.* APIs.

That was the case here, we were able to remove it: http://goo.gl/nBsmK4
This only affected some tests but we were able to solve the problem. As Robert 
said, it was on our TODO list already.
Tests sometimes want to assert special stuff. Production code in Lucene Core 
never used stuff like that.

> > Overall I like how this stuff is more locked down. I do think a lot of
> > projects will need to cleanup their code. For both these projects I am
> > testing, we already had a nice head start: testing 9-ea versions
> > already, running tests with security manager, trying to whittle down
> > the bad package accesses for a while now, submit fixes to libraries we
> > depend on, etc.
> Thanks for sending the mail with your experiences, this is exactly the type of
> discussion we are interested in.

Thanks for providing the builds before they are included into EA builds!

I will respond to the answer from Sundararajan Athijegannathan about the 
forbiddenapis static analysis tool, because this really causes headaches. For 
now we can no longer do the static analysis for our code quality, if you build 
Lucene with Java 9. The getResourceAsStream()/getResource() on JIGSAW packages 
always returning null is a disaster, also for many mocking frameworks or stuff 
that needs to "see" the bytecode in an easy way. Using the jrt-Filesystem makes 
the code horrible, because you have to mimic the whole classloader! And you 
cannot easily do that in tools that need to compile with Java 5 or Java 6 (most 
stuff in Maven/Ant has this requirement). :(

Uwe



Re: jigsaw EA feedback for apache lucene

2015-09-10 Thread Alan Bateman


On 10/09/2015 06:06, Robert Muir wrote:

:

Problems were basically all test/build related:
* API oddities around FileChannel required a better hack (thanks
Brian): 
http://mail.openjdk.java.net/pipermail/nio-dev/2015-September/003322.html
This one is subtle is that it was using core reflection to call a 
protected method in sun.nio.ch.FileChannelImpl. Static analysis with 
tools such as jdeps don't find cases like this. Good to see that a 
solution was found to this.



* Cleaning up a bunch of bad package accesses, these were already
TODOs in our test security policy, mostly just test bugs and the like.
"bad package accesses", do you mean direct use of JDK-internal classes? 
As you mention security policy then I will guess that these must be 
granting of accessClassInPackage.$PKG because the default policy when 
running with a security manager does not allow direct access to sun.* APIs.




:

Overall I like how this stuff is more locked down. I do think a lot of
projects will need to cleanup their code. For both these projects I am
testing, we already had a nice head start: testing 9-ea versions
already, running tests with security manager, trying to whittle down
the bad package accesses for a while now, submit fixes to libraries we
depend on, etc.
Thanks for sending the mail with your experiences, this is exactly the 
type of discussion we are interested in.


-Alan