Re: RFR: 8054717: SJavac should track changes in the public apis of classpath classes!
On 2015-03-03 12:53, Erik Joelsson wrote: That combination didn't actually build for me. When compiling jdk.jconsole, the following happened: java.lang.RuntimeException: com.sun.tools.javac.code.Symbol$CompletionFailure: class file for java.awt.datatransfer.UnsupportedFlavorException not found at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:143) at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:92) at com.sun.tools.sjavac.comp.SjavacImpl.compile(SjavacImpl.java:129) at com.sun.tools.sjavac.comp.PooledSjavac.lambda$compile$65(PooledSjavac.java:80) at com.sun.tools.sjavac.comp.PooledSjavac$$Lambda$4/703614162.call(Unknown Source) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:744) Caused by: com.sun.tools.javac.code.Symbol$CompletionFailure: class file for java.awt.datatransfer.UnsupportedFlavorException not found From what I can tell, jdk.jconsole uses java.desktop, which uses java.datatransfer (and re-exports it), where the missing class is. My best guess is that something in sjavac (API checking?) needs to traverse down into the transitive dependencies. This might be wrong, but as a workaround, it's easiest to fix in the makefiles so that we can have a working sjavac enabled jdk build. Here is an updated webrev which adds 3 levels (should cover everything I think) of transitive dependencies on the classpath when compiling modules: http://cr.openjdk.java.net/~erikj/8054717/webrev.root.02/ I'm not quite following this. This change affects the dependency calculation even when not using sjavac, right? Will that not cause any issues? What is the reasoning with the 3 levels? Is that the maximum depth currently in the JDK? Also, a minor nit, I thought we'd agreed to get rid of the $BUILD/tmp directory, and store intermediate build stuff in support or make-support. /Magnus
Re: RFR: 8054717: SJavac should track changes in the public apis of classpath classes!
On 2015-03-03 13:53, Magnus Ihse Bursie wrote: On 2015-03-03 12:53, Erik Joelsson wrote: That combination didn't actually build for me. When compiling jdk.jconsole, the following happened: java.lang.RuntimeException: com.sun.tools.javac.code.Symbol$CompletionFailure: class file for java.awt.datatransfer.UnsupportedFlavorException not found at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:143) at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:92) at com.sun.tools.sjavac.comp.SjavacImpl.compile(SjavacImpl.java:129) at com.sun.tools.sjavac.comp.PooledSjavac.lambda$compile$65(PooledSjavac.java:80) at com.sun.tools.sjavac.comp.PooledSjavac$$Lambda$4/703614162.call(Unknown Source) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:744) Caused by: com.sun.tools.javac.code.Symbol$CompletionFailure: class file for java.awt.datatransfer.UnsupportedFlavorException not found From what I can tell, jdk.jconsole uses java.desktop, which uses java.datatransfer (and re-exports it), where the missing class is. My best guess is that something in sjavac (API checking?) needs to traverse down into the transitive dependencies. This might be wrong, but as a workaround, it's easiest to fix in the makefiles so that we can have a working sjavac enabled jdk build. Here is an updated webrev which adds 3 levels (should cover everything I think) of transitive dependencies on the classpath when compiling modules: http://cr.openjdk.java.net/~erikj/8054717/webrev.root.02/ I'm not quite following this. This change affects the dependency calculation even when not using sjavac, right? Will that not cause any issues? What is the reasoning with the 3 levels? Is that the maximum depth currently in the JDK? I changed this to only affect SJAVAC_ENABLED builds. The implementation of FindTransitiveDependenciesForModule was taken from another branch where I had already needed it for a different reason. There, 3 levels happened to be enough and applying this patch solved the problem with sjavac, so I took it as it was for now. Also, a minor nit, I thought we'd agreed to get rid of the $BUILD/tmp directory, and store intermediate build stuff in support or make-support. Right, I moved it to support. New webrev: http://cr.openjdk.java.net/~erikj/8054717/webrev.root.03/ /Erik
Re: RFR: 8054717: SJavac should track changes in the public apis of classpath classes!
On 2015-03-03 15:03, Erik Joelsson wrote: On 2015-03-03 13:53, Magnus Ihse Bursie wrote: On 2015-03-03 12:53, Erik Joelsson wrote: That combination didn't actually build for me. When compiling jdk.jconsole, the following happened: java.lang.RuntimeException: com.sun.tools.javac.code.Symbol$CompletionFailure: class file for java.awt.datatransfer.UnsupportedFlavorException not found at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:143) at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:92) at com.sun.tools.sjavac.comp.SjavacImpl.compile(SjavacImpl.java:129) at com.sun.tools.sjavac.comp.PooledSjavac.lambda$compile$65(PooledSjavac.java:80) at com.sun.tools.sjavac.comp.PooledSjavac$$Lambda$4/703614162.call(Unknown Source) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:744) Caused by: com.sun.tools.javac.code.Symbol$CompletionFailure: class file for java.awt.datatransfer.UnsupportedFlavorException not found From what I can tell, jdk.jconsole uses java.desktop, which uses java.datatransfer (and re-exports it), where the missing class is. My best guess is that something in sjavac (API checking?) needs to traverse down into the transitive dependencies. This might be wrong, but as a workaround, it's easiest to fix in the makefiles so that we can have a working sjavac enabled jdk build. Here is an updated webrev which adds 3 levels (should cover everything I think) of transitive dependencies on the classpath when compiling modules: http://cr.openjdk.java.net/~erikj/8054717/webrev.root.02/ I'm not quite following this. This change affects the dependency calculation even when not using sjavac, right? Will that not cause any issues? What is the reasoning with the 3 levels? Is that the maximum depth currently in the JDK? I changed this to only affect SJAVAC_ENABLED builds. The implementation of FindTransitiveDependenciesForModule was taken from another branch where I had already needed it for a different reason. There, 3 levels happened to be enough and applying this patch solved the problem with sjavac, so I took it as it was for now. Also, a minor nit, I thought we'd agreed to get rid of the $BUILD/tmp directory, and store intermediate build stuff in support or make-support. Right, I moved it to support. New webrev: http://cr.openjdk.java.net/~erikj/8054717/webrev.root.03/ Thanks. I'm happy now! :-) /Magnus
Re: RFR: 8054717: SJavac should track changes in the public apis of classpath classes!
That combination didn't actually build for me. When compiling jdk.jconsole, the following happened: java.lang.RuntimeException: com.sun.tools.javac.code.Symbol$CompletionFailure: class file for java.awt.datatransfer.UnsupportedFlavorException not found at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:143) at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:92) at com.sun.tools.sjavac.comp.SjavacImpl.compile(SjavacImpl.java:129) at com.sun.tools.sjavac.comp.PooledSjavac.lambda$compile$65(PooledSjavac.java:80) at com.sun.tools.sjavac.comp.PooledSjavac$$Lambda$4/703614162.call(Unknown Source) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:744) Caused by: com.sun.tools.javac.code.Symbol$CompletionFailure: class file for java.awt.datatransfer.UnsupportedFlavorException not found From what I can tell, jdk.jconsole uses java.desktop, which uses java.datatransfer (and re-exports it), where the missing class is. My best guess is that something in sjavac (API checking?) needs to traverse down into the transitive dependencies. This might be wrong, but as a workaround, it's easiest to fix in the makefiles so that we can have a working sjavac enabled jdk build. Here is an updated webrev which adds 3 levels (should cover everything I think) of transitive dependencies on the classpath when compiling modules: http://cr.openjdk.java.net/~erikj/8054717/webrev.root.02/ /Erik On 2015-03-03 12:17, Erik Joelsson wrote: Hello, Here is my suggestion for makefile changes to go with the sjavac changes. Adding build-dev to get review for my part. Webrev: http://cr.openjdk.java.net/~erikj/8054717/webrev.root.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8054717 /Erik On 2015-03-02 13:31, Andreas Lundblad wrote: On Mon, Mar 02, 2015 at 09:42:58AM +0100, Erik Joelsson wrote: Sounds good to me. So you think I should do the makefile changes to adapt the JDK build to this patch in a separate bug? I'm not sure what's best, but I don't think we need a separate bug-entry. If you give me a pointer (or patch) on how to create/clean up a dummy empty bootclasspath directory, I can do the push. -- Andreas
Re: RFR: 8054717: SJavac should track changes in the public apis of classpath classes!
Hello, Here is my suggestion for makefile changes to go with the sjavac changes. Adding build-dev to get review for my part. Webrev: http://cr.openjdk.java.net/~erikj/8054717/webrev.root.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8054717 /Erik On 2015-03-02 13:31, Andreas Lundblad wrote: On Mon, Mar 02, 2015 at 09:42:58AM +0100, Erik Joelsson wrote: Sounds good to me. So you think I should do the makefile changes to adapt the JDK build to this patch in a separate bug? I'm not sure what's best, but I don't think we need a separate bug-entry. If you give me a pointer (or patch) on how to create/clean up a dummy empty bootclasspath directory, I can do the push. -- Andreas
Re: RFR 8054717: SJavac should track changes in the public apis of classpath classes!
It seems like the development speed for sjavac needs to be increased significantly, one JIRA bug, one 2 week review cycle, leads to one commit is much too slow. I belive it would be good to have a separate repository to do quicker development of sjavac, and test out all sorts of interesting stuff, after which a larger more polished review can be published here. I have now created such a high speed development repository and published my changes that I made over two late night hacks: https://bitbucket.org/jabberbeak/sjavac I believe that now when the OpenJDK sources are modularized, we should move sjavac to its own module perhaps jdk.sjavac. I also published a blog post on the contents of the high speed development repository, called The Smart Javac Wrapper JDK8 Backport with Extras and it can be found here: http://fredrikohrstrom.blogspot.se/ On Fri, Aug 22, 2014 at 8:42 PM, Jonathan Gibbons jonathan.gibb...@oracle.com wrote: I think the path to a thin Java client is relatively quick and easy. Once you have located the server, you just have to change the protocol to send argv over the connection and accept stream output coming back. The bulk of the code that currently lives in the fat client can equally well exist and be executed in the server. The bug that Eric described in his message of 08/21/2014 01:42 AM sounds as much like a bug in the build as in sjavac. If a file in a module is modified, in the worst case, all dependent modules should be recompiled. Yes, that is suboptimal, but at least it would give correct class files. sjavac may provide the means to reduce the set of modules that need to be recompiled, but the build should be capable of behaving reasonably even without sjavac. -- Jon On 08/22/2014 08:02 AM, Fredrik Öhrström wrote: The path to a super thin client is quite long with a lot of interesting technical problems (I suppose that is why the royal we has not authorized RFR:8044131 since that does not reach the thin client in one go, but is a mere first step) however classpath dependency tracking is needed now. 2014-08-22 0:05 GMT+02:00 Jonathan Gibbons jonathan.gibb...@oracle.com: Fredrik, Can you please explain the following some more: It is important that this public api scanning does not require a javac server started though. This is /not/ the direction we want to take sjavac. We want to change sjavac from its current client/server split to a (very) thin client, with most of the work happening in the server. In some version of an ideal world, the client would be so simple it could be written in C and not invoke a JVM at all. The client should just locate the server (starting it if necessary) and then dispatch its command line args to be processed in the server. Doing more and more work in a cold unshared JVM to avoid doing work in a hot shared JVM seems like a bad way to go. -- Jon On 08/21/2014 01:42 AM, Erik Joelsson wrote: I can't comment on the code quality of this patch, but I do think this feature is important. I can't recommend people to try sjavac with a straight face at the moment as it will no longer guarantee a correct incremental build. The current behavior when I add a public field to java/lang/Object is that all of java.base is recompiled, then the rest of the modules are sent to sjavac (as make thinks they need to be recompiled) and sjavac does nothing with them. /Erik On 2014-08-09 00:22, Fredrik Öhrström wrote: Here is very useful feature addition to sjavac, in fact it is required for a modularized OpenJDK build (Jigsawified), where not all sources are sent to sjavac at the same time. sjavac should track the public apis of the classes in the classpath, that the previous compile has linked to. If the public api of the linked to classes has changed, then this should trigger a recompile of the appropriate sources. This is a very useful feature. Currently sjavac must be fed all sources that belong to the product in a single huge compile. The OpenJDK is already at the limit of what fits in a reasonable (of today) sized Java heap. Other products are significantly larger and therefore cannot make use of the incremental compile in sjavac. With this feature, it is possible to compile the product as separate jar files, as long as the build dependencies for the jar files are a directed acyclic graph, then each succesive compile will detect if there were any changes in the public apis of the dependency jar files. If the public apis of the dependencies were not changed, and there were no other source changes for that jar, it will not be recompiled! But if there were changes in the public apis of the dependencies, then only the appropriate parts of the jar will be recompiled! This is the first draft of this large patch. In particular the new file Compile.java is a misnamer, it does NOT compile, it is used to extract the public apis of the classes on the classpath. Somehowe
