GraalVM's native-image incompatible with Clojure's eval?
Hi, I was doing a small experiment with Clojure and GraalVM and ended with this minimal reproduction case of an incompatibility between Clojure's `eval` and GraalVM's native-image tool (the program that compiles a JVM program to a native executable, based on the GraalVM's SubstrateVM compiler). Here's the Clojure program: (ns test-cli.main (:gen-class)) (set! *warn-on-reflection* true) (defn -main "I don't do a whole lot ... yet." [& args] (println (+ 1 1)) ;; trick to force loading clojure.lang.Numbers, not working (eval (read-string "(+ 1 1)"))) Using Clojure 1.9.0 and GraalVM version 1.0.0-rc9. When I compile it with the option "--report-unsupported-elements-at-runtime" (which gives a more precised error message), here's the output when I try executing the resulting executable: Exception in thread "main" java.lang.ClassNotFoundException: clojure.lang.Numbers, compiling:(NO_SOURCE_PATH:0:0) at java.lang.Throwable.(Throwable.java:287) at java.lang.Exception.(Exception.java:84) at java.lang.RuntimeException.(RuntimeException.java:80) at clojure.lang.Compiler$CompilerException.(Compiler.java:6804) at clojure.lang.Compiler.analyzeSeq(Compiler.java:7010) at clojure.lang.Compiler.analyze(Compiler.java:6773) at clojure.lang.Compiler.analyze(Compiler.java:6729) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6998) at clojure.lang.Compiler.analyze(Compiler.java:6773) at clojure.lang.Compiler.analyze(Compiler.java:6729) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6100) at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5460) at clojure.lang.Compiler$FnExpr.parse(Compiler.java:4022) at clojure.lang.Compiler.analyzeSeq(Compiler.java:7001) at clojure.lang.Compiler.analyze(Compiler.java:6773) at clojure.lang.Compiler.eval(Compiler.java:7059) at clojure.lang.Compiler.eval(Compiler.java:7025) at clojure.core$eval.invokeStatic(core.clj:3206) at test_cli.main$_main.invokeStatic(main.clj:7) at test_cli.main$_main.doInvoke(main.clj:7) at clojure.lang.RestFn.invoke(RestFn.java:397) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.RestFn.applyTo(RestFn.java:132) at test_cli.main.main(Unknown Source) at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:164) Caused by: java.lang.ClassNotFoundException: clojure.lang.Numbers at java.lang.Throwable.(Throwable.java:287) at java.lang.Exception.(Exception.java:84) at java.lang.ReflectiveOperationException.(ReflectiveOperationException.java:75) at java.lang.ClassNotFoundException.(ClassNotFoundException.java:82) at com.oracle.svm.core.hub.ClassForNameSupport.forName(ClassForNameSupport.java:51) at com.oracle.svm.core.hub.DynamicHub.forName(DynamicHub.java:1036) at clojure.lang.RT.classForName(RT.java:2204) at clojure.lang.RT.classForNameNonLoading(RT.java:2217) at clojure.lang.Compiler$HostExpr.maybeClass(Compiler.java:1041) at clojure.lang.Compiler$HostExpr$Parser.parse(Compiler.java:982) at clojure.lang.Compiler.analyzeSeq(Compiler.java:7003) ... 20 more I'm no expert in Java/JVM and would like to understand the problem. According to the SubstrateVM documentation (https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md) it can't compile Dynamic Class Loading/Unloading. Is Clojure's `eval` doing such dynamic loading? Or doing something else not supported by SubstrateVM as said in the documentation? Thanks *a lot* in advance for you answers. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: GraalVM's native-image incompatible with Clojure's eval?
Yes, eval will generate classes in a dynamic classloader, load them, then call methods on the newly formed class/object except for too-simple-to-be-interesting cases. On Mon, Nov 26, 2018 at 11:43 AM Khalid Jebbari wrote: > Hi, > > I was doing a small experiment with Clojure and GraalVM and ended with > this minimal reproduction case of an incompatibility between Clojure's > `eval` and GraalVM's native-image tool (the program that compiles a JVM > program to a native executable, based on the GraalVM's SubstrateVM > compiler). > > Here's the Clojure program: > > (ns test-cli.main > (:gen-class)) > > (set! *warn-on-reflection* true) > > (defn -main > "I don't do a whole lot ... yet." > [& args] > (println (+ 1 1)) ;; trick to force loading clojure.lang.Numbers, not > working > (eval (read-string "(+ 1 1)"))) > > Using Clojure 1.9.0 and GraalVM version 1.0.0-rc9. > > When I compile it with the > option "--report-unsupported-elements-at-runtime" (which gives a more > precised error message), here's the output when I try executing the > resulting executable: > > Exception in thread "main" java.lang.ClassNotFoundException: > clojure.lang.Numbers, compiling:(NO_SOURCE_PATH:0:0) > at java.lang.Throwable.(Throwable.java:287) > at java.lang.Exception.(Exception.java:84) > at java.lang.RuntimeException.(RuntimeException.java:80) > at > clojure.lang.Compiler$CompilerException.(Compiler.java:6804) > at clojure.lang.Compiler.analyzeSeq(Compiler.java:7010) > at clojure.lang.Compiler.analyze(Compiler.java:6773) > at clojure.lang.Compiler.analyze(Compiler.java:6729) > at clojure.lang.Compiler.analyzeSeq(Compiler.java:6998) > at clojure.lang.Compiler.analyze(Compiler.java:6773) > at clojure.lang.Compiler.analyze(Compiler.java:6729) > at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6100) > at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5460) > at clojure.lang.Compiler$FnExpr.parse(Compiler.java:4022) > at clojure.lang.Compiler.analyzeSeq(Compiler.java:7001) > at clojure.lang.Compiler.analyze(Compiler.java:6773) > at clojure.lang.Compiler.eval(Compiler.java:7059) > at clojure.lang.Compiler.eval(Compiler.java:7025) > at clojure.core$eval.invokeStatic(core.clj:3206) > at test_cli.main$_main.invokeStatic(main.clj:7) > at test_cli.main$_main.doInvoke(main.clj:7) > at clojure.lang.RestFn.invoke(RestFn.java:397) > at clojure.lang.AFn.applyToHelper(AFn.java:152) > at clojure.lang.RestFn.applyTo(RestFn.java:132) > at test_cli.main.main(Unknown Source) > at > com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:164) > Caused by: java.lang.ClassNotFoundException: clojure.lang.Numbers > at java.lang.Throwable.(Throwable.java:287) > at java.lang.Exception.(Exception.java:84) > at > java.lang.ReflectiveOperationException.(ReflectiveOperationException.java:75) > at > java.lang.ClassNotFoundException.(ClassNotFoundException.java:82) > at > com.oracle.svm.core.hub.ClassForNameSupport.forName(ClassForNameSupport.java:51) > at com.oracle.svm.core.hub.DynamicHub.forName(DynamicHub.java:1036) > at clojure.lang.RT.classForName(RT.java:2204) > at clojure.lang.RT.classForNameNonLoading(RT.java:2217) > at clojure.lang.Compiler$HostExpr.maybeClass(Compiler.java:1041) > at clojure.lang.Compiler$HostExpr$Parser.parse(Compiler.java:982) > at clojure.lang.Compiler.analyzeSeq(Compiler.java:7003) > ... 20 more > > I'm no expert in Java/JVM and would like to understand the problem. > According to the SubstrateVM documentation ( > https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md) > it can't compile Dynamic Class Loading/Unloading. Is Clojure's `eval` doing > such dynamic loading? Or doing something else not supported by SubstrateVM > as said in the documentation? > > Thanks *a lot* in advance for you answers. > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members
Re: GraalVM's native-image incompatible with Clojure's eval?
Thanks a lot. So it means Clojure's `eval` is by design incompatible with SubstrateVM. Does anyone know of others hard incompatibilities ? On Monday, November 26, 2018 at 6:16:20 PM UTC+1, Gary Trakhman wrote: > > Yes, eval will generate classes in a dynamic classloader, load them, then > call methods on the newly formed class/object except for > too-simple-to-be-interesting cases. > > On Mon, Nov 26, 2018 at 11:43 AM Khalid Jebbari > wrote: > >> Hi, >> >> I was doing a small experiment with Clojure and GraalVM and ended with >> this minimal reproduction case of an incompatibility between Clojure's >> `eval` and GraalVM's native-image tool (the program that compiles a JVM >> program to a native executable, based on the GraalVM's SubstrateVM >> compiler). >> >> Here's the Clojure program: >> >> (ns test-cli.main >> (:gen-class)) >> >> (set! *warn-on-reflection* true) >> >> (defn -main >> "I don't do a whole lot ... yet." >> [& args] >> (println (+ 1 1)) ;; trick to force loading clojure.lang.Numbers, not >> working >> (eval (read-string "(+ 1 1)"))) >> >> Using Clojure 1.9.0 and GraalVM version 1.0.0-rc9. >> >> When I compile it with the >> option "--report-unsupported-elements-at-runtime" (which gives a more >> precised error message), here's the output when I try executing the >> resulting executable: >> >> Exception in thread "main" java.lang.ClassNotFoundException: >> clojure.lang.Numbers, compiling:(NO_SOURCE_PATH:0:0) >> at java.lang.Throwable.(Throwable.java:287) >> at java.lang.Exception.(Exception.java:84) >> at java.lang.RuntimeException.(RuntimeException.java:80) >> at >> clojure.lang.Compiler$CompilerException.(Compiler.java:6804) >> at clojure.lang.Compiler.analyzeSeq(Compiler.java:7010) >> at clojure.lang.Compiler.analyze(Compiler.java:6773) >> at clojure.lang.Compiler.analyze(Compiler.java:6729) >> at clojure.lang.Compiler.analyzeSeq(Compiler.java:6998) >> at clojure.lang.Compiler.analyze(Compiler.java:6773) >> at clojure.lang.Compiler.analyze(Compiler.java:6729) >> at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6100) >> at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5460) >> at clojure.lang.Compiler$FnExpr.parse(Compiler.java:4022) >> at clojure.lang.Compiler.analyzeSeq(Compiler.java:7001) >> at clojure.lang.Compiler.analyze(Compiler.java:6773) >> at clojure.lang.Compiler.eval(Compiler.java:7059) >> at clojure.lang.Compiler.eval(Compiler.java:7025) >> at clojure.core$eval.invokeStatic(core.clj:3206) >> at test_cli.main$_main.invokeStatic(main.clj:7) >> at test_cli.main$_main.doInvoke(main.clj:7) >> at clojure.lang.RestFn.invoke(RestFn.java:397) >> at clojure.lang.AFn.applyToHelper(AFn.java:152) >> at clojure.lang.RestFn.applyTo(RestFn.java:132) >> at test_cli.main.main(Unknown Source) >> at >> com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:164) >> Caused by: java.lang.ClassNotFoundException: clojure.lang.Numbers >> at java.lang.Throwable.(Throwable.java:287) >> at java.lang.Exception.(Exception.java:84) >> at >> java.lang.ReflectiveOperationException.(ReflectiveOperationException.java:75) >> at >> java.lang.ClassNotFoundException.(ClassNotFoundException.java:82) >> at >> com.oracle.svm.core.hub.ClassForNameSupport.forName(ClassForNameSupport.java:51) >> at >> com.oracle.svm.core.hub.DynamicHub.forName(DynamicHub.java:1036) >> at clojure.lang.RT.classForName(RT.java:2204) >> at clojure.lang.RT.classForNameNonLoading(RT.java:2217) >> at clojure.lang.Compiler$HostExpr.maybeClass(Compiler.java:1041) >> at clojure.lang.Compiler$HostExpr$Parser.parse(Compiler.java:982) >> at clojure.lang.Compiler.analyzeSeq(Compiler.java:7003) >> ... 20 more >> >> I'm no expert in Java/JVM and would like to understand the problem. >> According to the SubstrateVM documentation ( >> https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md) >> it can't compile Dynamic Class Loading/Unloading. Is Clojure's `eval` doing >> such dynamic loading? Or doing something else not supported by SubstrateVM >> as said in the documentation? >> >> Thanks *a lot* in advance for you answers. >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this gr
Re: GraalVM's native-image incompatible with Clojure's eval?
Substrate makes a closed world assumption. jaotc is open world, and also based on Graal. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: GraalVM's native-image incompatible with Clojure's eval?
Thanks for pointing out jaotc. It's not what I need but it's good to know it exists. Do you know if it's possible to AOT (part of) a Clojure program with it? I'm curious. On Mon, Nov 26, 2018, 7:06 PM Ghadi Shayban Substrate makes a closed world assumption. jaotc is open world, and also > based on Graal. > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "Clojure" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/clojure/LMbNOg67wcw/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Prototype code for enhanced CIDER code completion for Java methods
Very cool! By the way, thanks for all the work you have put in to compliment. Sometimes it's hard to know when/how to thank people. I myself am all too often guilty of, "Thanks, can I have a new feature or bug fix?" You, Mr. Emerick, Mr. Batsov -- and many others -- thanks! I'd start a thanksgiving thread for clojure, but I'm afraid my list would be quite long, and sort of sound presumptuous, as if I'd accomplished something worthy of thanking the many folks whose work made it possible. So how about this -- Thanks to all the different folks in the community for sharing your work, whether I've used it or not, I've learned many things from: - Presentations by Rich Hickey - The Joy of Clojure - CIDER - Figwheel and ClojureScript - core.async, manifold, ring, sente, timbre, specter, seesaw, ... ‐‐‐ Original Message ‐‐‐ On Sunday, November 25, 2018 5:53 PM, Alexander Yakushev wrote: > To anyone who's interested in having precise completion for Java classes – a > feature like this is available in the latest CIDER snapshot: > https://twitter.com/unlog1c/status/1066748173094453248. You can get a > filtered completion by prepending a type tag to the next symbol. > > On Friday, October 12, 2018 at 3:22:37 AM UTC+3, > somewhat-functional-programmer wrote: > >> I'd like to share an idea and prototype code for better Java code completion >> in CIDER. While my main development environment is CIDER, the small >> modifications I made to support this idea were both to cider-nrepl and >> compliment -- which are both used by other Clojure tooling besides CIDER -- >> so maybe this is immediately more widely applicable. >> >> In an effort to make it easier on the tooling, I'm using a slightly >> different syntax for calling Java methods. My inspiration is Kawa scheme, >> and the notation is very similar: >> >> (String:charAt "my-string" 0) => \m >> (Integer:parseInt "12") => 12 >> (possibly.fully.qualified.YourClassName:method this args) >> >> For this syntax to be properly compiled of course it needs to be wrapped in >> a macro: >> >> One form: >> (jvm (String:charAt "my-string" 0)) >> >> Any number of forms: >> (jvm >> (lots of code) >> (JavaClass:method ...) >> (more code) >> (AnotherJavaClass:method ...)) >> >> The jvm macro will transform any symbol it finds in the calling position of >> a list that follows the ClassName:method convention. I was thinking maybe >> of limiting it to just a particular namespace to absolutely prevent any name >> collisions with real clojure functions, something like: >> >> (jvm/String:charAt "my-string" 0) >> >> This will also work with the one-off test code I'm including here for folks >> to see what they think. >> >> I actually like the syntax (though I wish I didn't have to wrap it in a jvm >> macro -- though if this actually idea was worth fully implementing, I'd >> imagine having new let or function macros so you don't even have to sprinkle >> "jvm" macros in code much at all). >> >> There is one additional advantages to this style of Java interop besides the >> far better code completion: >> - The jvm macro uses reflection to find the appropriate method at compile >> time, and as such, you get a compile error if the method cannot be found. >> - This is a downside if you *want* reflection, but this of course >> doesn't preclude using the normal (.method obj args) notation. >> >> You could even use this style for syntactic sugar for Java method handles: >> - Though not implemented in my toy code here, you could also pass >> String:charAt as a clojure function -- assuming there were no overloads of >> the same arity. >> >> So, I'm hoping you will try this out. Two things to copy/paste -- one is a >> boot command, the other is the 100-200 lines of clojure that implements a >> prototype of this. >> >> This command pulls the necessary dependencies as well as starts up the >> rebel-readline repl (which is fantastic tool, and it also uses compliment >> for code completion): >> >> # Run this somewhere where you can make an empty source directory, >> # something fails in boot-tools-deps if you don't have one >> # (much appreciate boot-tools-deps -- as cider-nrepl really needs to >> #be a git dep for my purpose here since it's run through mranderson for >> its normal distro) >> mkdir src && \ >> boot -d seancorfield/boot-tools-deps:0.4.6 \ >> -d compliment:0.3.6 -d cider/orchard:0.3.1 \ >> -d com.rpl/specter:1.1.1 -d com.taoensso/timbre:4.10.0 \ >> -d com.bhauman/rebel-readline:0.1.4 \ >> -d nrepl/nrepl:0.4.5 \ >> deps --config-data \ >> '{:deps {cider/cider-nrepl {:git/url >> "https://github.com/clojure-emacs/cider-nrepl.git"; :sha >> "b2c0b920d762fdac2f8210805df2055af63f2eb1"}}}' \ >> call -f rebel-readline.main/-main >> >> Paste the following code into the repl: >> >> (require 'cider.nrepl.middleware.info) >> >> (ns java-interop.core >> (:require >>[taoensso.timb
Re: Question: How do you organize large clojure projects?
Thank you, it does help. I almost went the monorepo route but for some reason thought I'd have one project.clj file with one giant list of dependencies. Having a monorepo but with different dependency sets makes a ton of sense and I think would have been much easier to manage. ‐‐‐ Original Message ‐‐‐ On Thursday, November 15, 2018 7:12 AM, Patrik Sundberg wrote: > I've been happy using a monorepo with boot, and a build.boot with many > building blocks that can be mixed and matched for many deployables (uberjars > in my case). > > In my build.boot I define my internal blocks, but also defs for external deps > like eg postgres and grpc. That way I have one place for these groups of > dependencies to manage, and can merge them into many of my own building > blocks in a flexible and consistent way. > > There are no checkouts, multiple outputs and consistent usage of dependencies. > > If I need different release management/deployment process for different > deployables of my own I tend to use different release branches by deployable > that my ci/CD setup triggers from. > > Hope that helps! > > - > > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > > > > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: GraalVM's native-image incompatible with Clojure's eval?
It's already possibly, but not really part of, that would make little sense anyway. It shines with tools you want to run quickly, where the reduction in startup time helps. There was a tweet if a formatter compiled to a native image. I also tried to create a native image for a pretty project, but turned out there are unresolved issues with the Java Kafka Client. Another place where it can be great is really scalable micro services and/or use them as serverless functions. But you lose yit optimisations. So peak performance probably is better running on a JVM. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.