Re: Binary compatibility and a culture of AOT abuse
On Tue, Dec 15, 2009 at 7:05 AM, Phil Hagelberg wrote: > Yep, this will be changed in the next release of leiningen. The default > build will only AOT namespaces that actually need it to > function. Naturally you will still be able to build jars that AOT > everything, but you will have to specifically ask for that behaviour. I also have an almost working polyglot maven3 clojure module working as well (reads and writes maven poms in clojure using a syntax initially borrowed from leiningen but extended for the rest of mavens bits and pieces - trying to sort out the syntax I want to use is annoyingly hard!). The way I was thinking of doing it was configuring up a default assembly plugin instance to build the slim jar, a clojure:slimjar goal on the clojure-maven-plugin ( probably the better place to put it ). -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Binary compatibility and a culture of AOT abuse
B Smith-Mannschott writes: > This issue has also been brought up in connection with leiningen, > which currently AOT-compiles everything as part of its normal build. > My impression is that it would be smarter to be selective about what > gets AOT compiled, and what doesn't. Yep, this will be changed in the next release of leiningen. The default build will only AOT namespaces that actually need it to function. Naturally you will still be able to build jars that AOT everything, but you will have to specifically ask for that behaviour. -Phil -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Binary compatibility and a culture of AOT abuse
On Dec 14, 2009, at 7:16 AM, B Smith-Mannschott wrote: >> Then, to depend upon only the source jar in a downstream project, one >> would simply add a 'classifier' element to the dependency element: >> >> >>com.ashafa >>clutch >>1.0-SNAPSHOT >>sources >> > > Hmmm... but in the Clojure world there's a third variant, isn't there? > i.e. something like 'slim' which contains AOT only where required: > namely in the case of genclass or such. Yes, that's true. This is something that lein and clojure-maven- plugin et al. will have to consider going forward. - Chas -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Binary compatibility and a culture of AOT abuse
2009/12/14 Meikel Brandmeyer > Hi Laurent, > > Am 14.12.2009 um 09:43 schrieb Laurent PETIT: > > > But from what I (currently) know, AOTing jars should be harmless. > > So maybe the problem is that there's a bug in the new branch, and this > bug needs to be corrected ? > > No. The inner workings of Clojure might change. AOT compiled code doesn't > know these changes. I had this with some change to defmulti a while ago. > However the source is always recompiled when it's loaded. So it always see > the change and all is fine. (Of course only, as long as the change doesn't > involve a public API.) > > So i's a kind of break of compatibility with AOT compiled code. That should be noted in future official clojure releases, indeed, since it requires every library to be recompiled. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Binary compatibility and a culture of AOT abuse
Hi Laurent, Am 14.12.2009 um 09:43 schrieb Laurent PETIT: > But from what I (currently) know, AOTing jars should be harmless. > So maybe the problem is that there's a bug in the new branch, and this bug > needs to be corrected ? No. The inner workings of Clojure might change. AOT compiled code doesn't know these changes. I had this with some change to defmulti a while ago. However the source is always recompiled when it's loaded. So it always see the change and all is fine. (Of course only, as long as the change doesn't involve a public API.) Sincerely Meikel smime.p7s Description: S/MIME cryptographic signature
Re: Binary compatibility and a culture of AOT abuse
On Mon, Dec 14, 2009 at 11:31, Chas Emerick wrote: > There are certainly binary incompatibility issues between the > different versions/branches -- that'll settle out as the core matures, > and IIUC, especially once c-in-c becomes a reality. > > However, only providing libraries as source (non-AOT-compiled) jars > whenever possible only shifts the problem around. Interop > requirements are often known of ahead of time (ha!), but many > "classloader issues" are totally in the hands of whomever is deploying > the end application, their java security policy, etc. In those > contexts, having only clojure source libraries available in maven > repos is an active hurdle to usage perhaps as irritating as having to > unwind "unnecessary" AOT compilation. Of course, people with a strict > AOT-compilation requirement can do the AOT-compilation on their own, > but I for one do not want to see Gentoo-esque norms emerge in the > Clojure community. > > A more general solution would be to encourage library authors to > produce and deploy both source *and* binary, AOT-compiled artifacts, > and leave the choice to the library's users. I set this up on > Saturday for clutch (a clojure couchdb library and view server) in the > process of mavenizing the project: > > http://github.com/cemerick/clutch/commit/aef461a6fd3c4797aef3b110677197f7351bb831 > > Then, to depend upon only the source jar in a downstream project, one > would simply add a 'classifier' element to the dependency element: > > > com.ashafa > clutch > 1.0-SNAPSHOT > sources > Hmmm... but in the Clojure world there's a third variant, isn't there? i.e. something like 'slim' which contains AOT only where required: namely in the case of genclass or such. > I think offering both types of artifacts is a much more user-friendly > approach, is *very* easy to do (assuming maven usage), and resolves > all of the issues in play here (modulo the clojure.lang binary > signature incompatibilities that exist, but those should resolve > themselves in due course). > > Cheers, > > - Chas > > On Dec 13, 5:44 pm, dysinger wrote: >> So in my experiments with using clojure / contrib w/ the "new" branch, >> I've noticed a pattern of binary incompatibility. Jars pushed to >> clojars, maven repos and other places that are are unnecessarily >> AOTed, don't work with clojure "new". I noticed the same thing going >> from clojure 1.0 to clojure 1.1.0-SNAP. >> >> It doesn't really seem to matter if you include the src "clj" files >> along with the class files in your jar. As soon as clojure hits a >> classfile with an older AOTed namespace, it blows up. This isn't a >> criticism of clojure. Clojure needs it's freedom to innovate. >> >> ...but clojure, by default, JIT compile clj files. I would just like >> to start a conversation about using AOT sparingly if you publish your >> clojure project as a library. Ask yourself "why am I AOTing this >> namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a >> gen-class I need outside clojure in some fancy XML config file? Is >> there some classloader issue that makes it necessary to compile AOT ?" >> If the answer is NO, then please don't AOT the code. >> >> I went through the pain of 3 days of on/off twiddling with libraries >> so our project could use clojure "new". The hair-pulling details >> where in all the code that was AOT when there was no reason to do >> this. 99% of time there were no gen-class or anything else to require >> it. People just like compiling stuff [ I get the feeling they think it >> has some sort of ricer benefit to AOT compile ]. Most clj files work >> great with all versions of clojure if you just jar the clj files and >> leave class generation for runtime. >> >> Don't get me wrong, there are places where a single namespace or two >> are necessarily AOTed for interop or for classloader issues. But just >> carte blanche AOT makes it harder on other people using your library - >> to the point of trying to remove your library from their dependencies >> (trust me - been there). >> >> Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new" >> are restricted to one version when AOTed and 99% for no reason. >> >> Maybe Rich can comment on where I am missing it. But I think this is >> my stance. >> >> -Tim > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first pos
Re: Binary compatibility and a culture of AOT abuse
There are certainly binary incompatibility issues between the different versions/branches -- that'll settle out as the core matures, and IIUC, especially once c-in-c becomes a reality. However, only providing libraries as source (non-AOT-compiled) jars whenever possible only shifts the problem around. Interop requirements are often known of ahead of time (ha!), but many "classloader issues" are totally in the hands of whomever is deploying the end application, their java security policy, etc. In those contexts, having only clojure source libraries available in maven repos is an active hurdle to usage perhaps as irritating as having to unwind "unnecessary" AOT compilation. Of course, people with a strict AOT-compilation requirement can do the AOT-compilation on their own, but I for one do not want to see Gentoo-esque norms emerge in the Clojure community. A more general solution would be to encourage library authors to produce and deploy both source *and* binary, AOT-compiled artifacts, and leave the choice to the library's users. I set this up on Saturday for clutch (a clojure couchdb library and view server) in the process of mavenizing the project: http://github.com/cemerick/clutch/commit/aef461a6fd3c4797aef3b110677197f7351bb831 Then, to depend upon only the source jar in a downstream project, one would simply add a 'classifier' element to the dependency element: com.ashafa clutch 1.0-SNAPSHOT sources I think offering both types of artifacts is a much more user-friendly approach, is *very* easy to do (assuming maven usage), and resolves all of the issues in play here (modulo the clojure.lang binary signature incompatibilities that exist, but those should resolve themselves in due course). Cheers, - Chas On Dec 13, 5:44 pm, dysinger wrote: > So in my experiments with using clojure / contrib w/ the "new" branch, > I've noticed a pattern of binary incompatibility. Jars pushed to > clojars, maven repos and other places that are are unnecessarily > AOTed, don't work with clojure "new". I noticed the same thing going > from clojure 1.0 to clojure 1.1.0-SNAP. > > It doesn't really seem to matter if you include the src "clj" files > along with the class files in your jar. As soon as clojure hits a > classfile with an older AOTed namespace, it blows up. This isn't a > criticism of clojure. Clojure needs it's freedom to innovate. > > ...but clojure, by default, JIT compile clj files. I would just like > to start a conversation about using AOT sparingly if you publish your > clojure project as a library. Ask yourself "why am I AOTing this > namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a > gen-class I need outside clojure in some fancy XML config file? Is > there some classloader issue that makes it necessary to compile AOT ?" > If the answer is NO, then please don't AOT the code. > > I went through the pain of 3 days of on/off twiddling with libraries > so our project could use clojure "new". The hair-pulling details > where in all the code that was AOT when there was no reason to do > this. 99% of time there were no gen-class or anything else to require > it. People just like compiling stuff [ I get the feeling they think it > has some sort of ricer benefit to AOT compile ]. Most clj files work > great with all versions of clojure if you just jar the clj files and > leave class generation for runtime. > > Don't get me wrong, there are places where a single namespace or two > are necessarily AOTed for interop or for classloader issues. But just > carte blanche AOT makes it harder on other people using your library - > to the point of trying to remove your library from their dependencies > (trust me - been there). > > Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new" > are restricted to one version when AOTed and 99% for no reason. > > Maybe Rich can comment on where I am missing it. But I think this is > my stance. > > -Tim -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Binary compatibility and a culture of AOT abuse
Hello, Interesting return of experience. But from what I (currently) know, AOTing jars should be harmless. So maybe the problem is that there's a bug in the new branch, and this bug needs to be corrected ? 2009/12/13 dysinger > So in my experiments with using clojure / contrib w/ the "new" branch, > I've noticed a pattern of binary incompatibility. Jars pushed to > clojars, maven repos and other places that are are unnecessarily > AOTed, don't work with clojure "new". I noticed the same thing going > from clojure 1.0 to clojure 1.1.0-SNAP. > > It doesn't really seem to matter if you include the src "clj" files > along with the class files in your jar. As soon as clojure hits a > classfile with an older AOTed namespace, it blows up. This isn't a > criticism of clojure. Clojure needs it's freedom to innovate. > > ...but clojure, by default, JIT compile clj files. I would just like > to start a conversation about using AOT sparingly if you publish your > clojure project as a library. Ask yourself "why am I AOTing this > namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a > gen-class I need outside clojure in some fancy XML config file? Is > there some classloader issue that makes it necessary to compile AOT ?" > If the answer is NO, then please don't AOT the code. > > I went through the pain of 3 days of on/off twiddling with libraries > so our project could use clojure "new". The hair-pulling details > where in all the code that was AOT when there was no reason to do > this. 99% of time there were no gen-class or anything else to require > it. People just like compiling stuff [ I get the feeling they think it > has some sort of ricer benefit to AOT compile ]. Most clj files work > great with all versions of clojure if you just jar the clj files and > leave class generation for runtime. > > Don't get me wrong, there are places where a single namespace or two > are necessarily AOTed for interop or for classloader issues. But just > carte blanche AOT makes it harder on other people using your library - > to the point of trying to remove your library from their dependencies > (trust me - been there). > > Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new" > are restricted to one version when AOTed and 99% for no reason. > > Maybe Rich can comment on where I am missing it. But I think this is > my stance. > > -Tim > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Binary compatibility and a culture of AOT abuse
Hi, I like to suggest the following policy: publish libraries as jarred- up .clj sources and AOT them locally when building an application. Leiningen could sure help here and everen cache different compiled versions dependening on the clojure version. IIRC this is what Debian does with elisp packages. Philipp Meier -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Binary compatibility and a culture of AOT abuse
On Mon, Dec 14, 2009 at 07:26, mac wrote: > On Dec 13, 11:44 pm, dysinger wrote: >> So in my experiments with using clojure / contrib w/ the "new" branch, >> I've noticed a pattern of binary incompatibility. Jars pushed to >> clojars, maven repos and other places that are are unnecessarily >> AOTed, don't work with clojure "new". I noticed the same thing going >> from clojure 1.0 to clojure 1.1.0-SNAP. >> >> It doesn't really seem to matter if you include the src "clj" files >> along with the class files in your jar. As soon as clojure hits a >> classfile with an older AOTed namespace, it blows up. This isn't a >> criticism of clojure. Clojure needs it's freedom to innovate. >> >> ...but clojure, by default, JIT compile clj files. I would just like >> to start a conversation about using AOT sparingly if you publish your >> clojure project as a library. Ask yourself "why am I AOTing this >> namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a >> gen-class I need outside clojure in some fancy XML config file? Is >> there some classloader issue that makes it necessary to compile AOT ?" >> If the answer is NO, then please don't AOT the code. >> >> I went through the pain of 3 days of on/off twiddling with libraries >> so our project could use clojure "new". The hair-pulling details >> where in all the code that was AOT when there was no reason to do >> this. 99% of time there were no gen-class or anything else to require >> it. People just like compiling stuff [ I get the feeling they think it >> has some sort of ricer benefit to AOT compile ]. Most clj files work >> great with all versions of clojure if you just jar the clj files and >> leave class generation for runtime. >> >> Don't get me wrong, there are places where a single namespace or two >> are necessarily AOTed for interop or for classloader issues. But just >> carte blanche AOT makes it harder on other people using your library - >> to the point of trying to remove your library from their dependencies >> (trust me - been there). >> >> Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new" >> are restricted to one version when AOTed and 99% for no reason. >> >> Maybe Rich can comment on where I am missing it. But I think this is >> my stance. >> >> -Tim > > I've never really used many clojure libs at the same time or even any > very big ones so maybe I'm mistaken here, but does not AOT compilation > help startup time? Yes, it does help considerably with startup time. I got into the habit of AOT compiling my stuff because my primary development machine has a 1.6GHz Atom and starting up from "slim" builds was too slow for my taste. That said, I think Tim's point is well taken. I should back away from AOT compiling again and see if the performance is still as much of a problem as I remember. This issue has also been brought up in connection with leiningen, which currently AOT-compiles everything as part of its normal build. My impression is that it would be smarter to be selective about what gets AOT compiled, and what doesn't. // Ben > I know everyone is all internetty and webby these days but I kinda > like desktop apps myself and those need to start and stop a lot more > than server apps, which makes such things important. > Also from a developer point of view working with server software it's > nice if you can restart things quickly when the system has gotten into > a bad state because you carelessly deployed something broken on your > dev machine. > > But if the time difference is not noticeable, then I certainly agree > with you for open source projects. > > /Mac -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Binary compatibility and a culture of AOT abuse
On Dec 13, 11:44 pm, dysinger wrote: > So in my experiments with using clojure / contrib w/ the "new" branch, > I've noticed a pattern of binary incompatibility. Jars pushed to > clojars, maven repos and other places that are are unnecessarily > AOTed, don't work with clojure "new". I noticed the same thing going > from clojure 1.0 to clojure 1.1.0-SNAP. > > It doesn't really seem to matter if you include the src "clj" files > along with the class files in your jar. As soon as clojure hits a > classfile with an older AOTed namespace, it blows up. This isn't a > criticism of clojure. Clojure needs it's freedom to innovate. > > ...but clojure, by default, JIT compile clj files. I would just like > to start a conversation about using AOT sparingly if you publish your > clojure project as a library. Ask yourself "why am I AOTing this > namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a > gen-class I need outside clojure in some fancy XML config file? Is > there some classloader issue that makes it necessary to compile AOT ?" > If the answer is NO, then please don't AOT the code. > > I went through the pain of 3 days of on/off twiddling with libraries > so our project could use clojure "new". The hair-pulling details > where in all the code that was AOT when there was no reason to do > this. 99% of time there were no gen-class or anything else to require > it. People just like compiling stuff [ I get the feeling they think it > has some sort of ricer benefit to AOT compile ]. Most clj files work > great with all versions of clojure if you just jar the clj files and > leave class generation for runtime. > > Don't get me wrong, there are places where a single namespace or two > are necessarily AOTed for interop or for classloader issues. But just > carte blanche AOT makes it harder on other people using your library - > to the point of trying to remove your library from their dependencies > (trust me - been there). > > Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new" > are restricted to one version when AOTed and 99% for no reason. > > Maybe Rich can comment on where I am missing it. But I think this is > my stance. > > -Tim I've never really used many clojure libs at the same time or even any very big ones so maybe I'm mistaken here, but does not AOT compilation help startup time? I know everyone is all internetty and webby these days but I kinda like desktop apps myself and those need to start and stop a lot more than server apps, which makes such things important. Also from a developer point of view working with server software it's nice if you can restart things quickly when the system has gotten into a bad state because you carelessly deployed something broken on your dev machine. But if the time difference is not noticeable, then I certainly agree with you for open source projects. /Mac -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
