Hi Daniel, the Gradle dev team has synced their holidays :) I'm back in my office on Friday and I'm looking forward to replying to your email then.
- Hans On Wed, 08 Jul 2009 12:51 +0700, "Daniel" <[email protected]> wrote: > On Wed, Jul 8, 2009 at 11:28 AM, Phil Hagelberg<[email protected]> wrote: > > > > I've been noodling on the problem of dependency management for a while > > now. It's definitely a pain point for projects with more than a couple > > dependencies. Currently our approach has been to use maven, but that > > involves a fair amount of arcane knowledge as well as writing a bunch of > > XML, which isn't a lot of fun. But there's been a community consensus > > that any dependency tool needs to be able to leverage all the valuable > > stuff out there that's currently in maven repositories. > > > > I've put together a proof-of-concept dependency manager called > > Corkscrew. It uses Maven under the hood as well as offering dependencies > > on source packages stored in git or subversion. A simple project.clj > > file lays out the dependencies and other project info: > > > > {:name "my-sample" > > :version "1.0" > > :dependencies [["tagsoup" "1.2" "org.ccil.cowan.tagsoup"] > > ;; group defaults to name > > ["rome" "0.9"]] > > :source-dependencies [["clojure-contrib" "r663" :svn > > > > "http://clojure-contrib.googlecode.com/svn/trunk";] > > ["enlive" > > "95b2558943f50bb9962fe7d500ede353f1b578f0" > > :git "git://github.com/cgrand/enlive.git"]]} > > > > You can install it with: > > > > $ git clone git://github.com/technomancy/corkscrew.git > > $ cd corkscrew > > $ ./install > > $ cp bin/corkscrew /somewhere/on/your/path > > > > Create a project.clj file in your project root based on the one > > above. Then you can use "corkscrew deps" to set everything up for > > you. At that point just make sure your classpath includes > > target/dependency/ and you should be good to go. > > > > It's simple, but it solves the main problems that I've been having with > > more complicated projects. I'd love to get some opinions on it. > > > > The biggest issue right now is that it runs Maven as a subprocess rather > > than using the Java API in the same VM because I can't make head or tail > > of the Maven Java API (it uses plexus.core), but shelling out works as a > > proof-of-concept even if it's tacky. > > > > -Phil > > > > Thanks for bringing that up, as it's been bugging me for a some time > now (and violently ejected me out of lurk mode on the list). What > follows is a pretty long essay on the topic, I hope you stay with me, > and take part in the debate, because I think the availability of a > good buildtool/dependency manager is absolutely crucial to use Clojure > in larger projects, and as such for adoption on a wider scale. > > I have basically the same problem as you in that Clojure is lacking a > proper buildtool (i.e. one that can replace Maven). I used Maven for a > long time and lived with it's strengths and weaknesses. It's far from > perfect, but Maven does still give you quite a lot: > * a dependency resolution and publishing model for generated artifacts > * a build based on conventions (can be overridden, but Maven pushes > you hard to do things in a de-facto standard way) > * a defined execution model (the phases thing. Not always nice, and > not always sufficient, but at least something) > * a defined API to hook into with your plugins (API design is so-so, > and documentation is IMHO abysmal, but ...) > * a ton of reports and plugins to extend the model (not always > perfect, and configuration is not nice, but they usually do what they > advertise) > * more? > > On the other hand, some things Maven are always a pain in the back: > figuring out how to configure plugins, writing plugins because it's > the only way to do something non-standard in Maven, and > debugging/finding out what the heck is going on, if some plugin is not > doing what it should do... That's the arcane knowledge part. > > But recently I had the need for a more flexible buildtool that would > allow me to build non-standard JVM based languages in a non-standard > fashion and still play nice. Case in point was that I had to marry two > different dependency resolution systems (Maven style, and > Jython/Python style), and then build a mix of code in > Scala/Java/Jython. The experience was none too pretty with Maven. > > Most of the buildtools out there for JVM related stuff don't try to > compete with Maven and all it's plugins and reports, but they start by > replacing/wrapping Ant (and usually Ivy for dependency management). > That description applies to Buildr, the Groovy Ant Builder, Gant, > Grape (which is a wrapper for Ivy), SBT (Simple Build Tool - Scala), > Scalr (Scala), Lancet, and probably some more that I missed. The only > notable exception I could dig up was Gradle. > > Gradle (http://gradle.org/) turns out to be very interesting. Here are > some propaganda points (not exhaustive): > * Although it currently uses mainly a Groovy DSL, the Gradle core is > written in Java, and there's an explicit goal on the roadmap to > support DSLs written in other JVM languages on top of it as to make it > an attractive choice for everybody. For Scala there's a patch in the > Bugtracker to add a Scala DSL on top. Other languages I'm unaware of. > * Convention based builds. They have taken care that they appeal to > Maven developers (all the conventions of the Java Plugin are the same > ones as in Maven), while still leaving it flexible through > configuration that every project layout can be built. > * Hierarchical, dependency based builds. > * A build is executed in two major phases (configuration and > execution), and tasks are allowed to hook into both (a task can make > it's execution dependent on whether another task is scheduled to run > *after* itself (try that with Maven)) > * A build graph containing all the tasks for execution can be accessed > and manipulated programmatically. > * Gradle defines it's own task API, but provides an Ant wrapper to > give you an easy migration route or extension path (similar to Gant, > or Lancet in Clojure). Custom tasks can be easily coded up, although > you need custom things less often than with Maven and Co., because a > lot can be solved with a flexible build script (it's all code). > * The build can be extended from within a project. You don't need a > plugin project that you're only going to use in one project anyway. > There's a separate source tree in your project home that hosts > extension to your build. > * Dependency resolution is done with Ivy. But they took care to clean > the Ivy model a bit up so that it provides better support for > convention based builds. > * They have an integration layer with Maven. I don't know how far it > actually goes, but I know that you can call through to Maven for some > things (MavenPlugin). > > Non-technical > * The mailinglist is very active, and receptive to input that comes > from the outside. They have github mirrors, pull requests can be > issued easily. > * The project is well setup: toplevel domain, liberal license (Apache > v2), plethora of SCM can be used (master is on subversion, mirrors on > github (git), and launchpad (bzr)) issue tracker (JIRA), wiki > (Confluence), public roadmap, continuous integration (bamboo & > teamcity, platform specific builds), selfhosting (gradle is built by > gradle), documentation is autogenerated during the build (including > the userguide HTML and PDFs). > * The guys working on Gradle have been around the build tool circus > for a long time apparently (five committers, among them: the founder > of JBoss IDE, and the author of Gant). > * The team is dedicated enough to evangelize their product largescale, > including talks on conferences around the globe, and at least one of > the participants runs a consultancy providing advice on buildsystems. > * Reasonably complete documentation (there's arguably room for > improvement, especially in structure, but you can find almost anything > in the userguide). > * They are willing to change stuff in how it works internally if > there's technical merit. > * It's developing fast, and I have the feeling that it's getting > increasingly powerful and flexible. > * It's not very complicated IMHO. It's probably slightly more > complicated than ant + sugar coating, but they Gradle tagline is: > "make the impossible possible, the possible easy, and the easy > elegant." (Moshé Feldenkrais), and I think they are definitely within > scope. > > > Ok, now the question why am I pushing this so hard? Because have been > annoyed at the exact same things in every language that I tried for > the JVM. Maven is not very good but it's usually the best thing you > can get. And although there's a lot of competition for build tools, > nobody seems to try to tackle what Maven solved, and only goes for a > simpler subset of the problems. Usually it goes something like: "Our > language is more productive than Java and we don't like Maven, because > it's a pain to use. Ant is nice but uses XML so we can't use it out of > the box. We can't really wrap Maven, so let's wrap Ant, and since we > are lacking dependency management in that case, let's also wrap Ivy, > add a little bit of sugar on top, and voila, new shiny build system." > Except that everybody now has to learn a gazillion new (and different) > tools if he wants to build stuff easily in a certain language, and > beware if you try to do something cross language (a problem that will > become more common with alternative languages for the JVM on the > rise). > > I would very much like to see that people don't suffer the NIH > syndrome everywhere. Usually people forget that every alternative > technology implemented has an associated cost that everyone looking > for a solution has to pay, because I have to consciously discard the > tool as insufficient, and that means I have to acquire enough > knowledge about each tool to make an informed decision (the paradox of > choice). I cannot even imagine how much time has been wasted by > halfbaked (and partially or fully abandoned) projects. > > If the miracle would happen and the different, rapidly growing > language communities on the JVM finally could commit themselves to > support and push a shared buildtool, then that tool would have enough > steam to grow into a decent alternative for Maven. It might even > become the preferred way of build at least non-Java projects, who > knows. If everybody is trying to get their own share of followers and > only solves what is immediately visible for the projects used by > themselves and their followers, then we will never get anywhere. > > And please note that I'm not asking to start from scratch. Gradle is > there and has momentum already. I'm just hoping that I'm not the only > one that sees a benefit in joining forces with stuff that already > exists. If people in the Clojure community start to throw your weight > behind something that seems to be on the right track, can still be > influenced and made better. > > To get started with Gradle, you would have to: > * write a DSL that can talk to Gradle Java API. > * write a ClojurePlugin (probably based on the JavaPlugin) for Gradle, > if you have specialized wishes (such as that src/main/clojure should > be the default convention for Clojure projects, or that you can > specify which version of Clojure should be used for compilation etc.). > > </propaganda> > > Sorry for the long mail/rant, but it's been too painful too many > times. And a sincere apology to Phil, who just scratched an itch for > himself, and was interested in sharing it with the larger Clojure > community. This mail is not an attack on your work, or approach. It > just triggered something that I have stumbled over too many times and > I would love to see it finally resolved. I think that Gradle could be > used to give you all you envisioned for Corkscrew, but in the process, > you will gain so much more. > > Disclaimer: I'm (currently) not actively developing Gradle itself, but > i was impressed with what's there and where it's going. I also took > the liberty to crosspost this mail to the Gradle dev list (CC'd) to > invite a broader participation. > > I'm really hoping that we could find some consensus on the topic. > > These are my probably 20$ or so... > > Daniel > (probably by now unofficial Gradle Evangelist) > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
