Reasons why Java 7 got me excited about Ant 2
I thought I'd expand on why I got so excited about NIO 2.0 and JSR 203 as well as the other changes in Java 7. I can see I have done a bad job of selling the benefits of a refactoring for potential volunteers. Reading through the changes that were introduced in Java 7, it seems to me that Ant was probably an inspiration for a lot of them. Others come from taking the performance of I/O on Java more seriously. This has the potential to speed Ant up enormously. Projects like Tim Bray's Wide Finder 2[1] showcased just how badly Java performed and benchmarks like that may have been an impetus to focus resources on taking advantage of all the features of the native filesystems. Unfortunately the results of the project are lost due to Oracle's acquisition of Sun, but you can still see the results from the original Wide Finder 1[2] although there isn't even an attempt to do a Java implementation there. I can tell you from memory that the final results on WF2 were several C implementations with innovative I/O using parallel processing at the top, erlang and ruby and scala were up there, but the fastest Java implementation was in the double digits in the list as I recall. The basics for file handling in Java 7 are provided by Path and the Paths Utility class, along with the FileSystemProvider classes. If Ant 2 were created to make Resource a fundamental element and Path the Resource that represented something on a file system (native file systems, zip/jar file systems, and I expect eventually open source projects will add others like HTTP, FTP, NFS and Samba), I think the LOC count on Ant could be reduced tremendously. Writing patches would be simplified and contributing to Ant would have a lower barrier to entry for new people coming in. Path is immutable which suits Ant well. Path has methods to access elements in a file path, compare, test, normalize, and relativize paths in the ways that Ant has code to do. It has all kinds of features for determining accessibility on the native filesystem and bulk loading file metadata attributes whereas currently it is done one field at a time. Symbolic link identification and handling is a core feature. And converting to and from File objects is trivial. Add to that the new methods on File. There is also a Files utility helper class. Files covers the most basic tasks in Ant implemented using all the features of the native filesystem to make them as fast as possible: copying; deleting; moving; reading file contents; writing; creating directories, file and links; getting input and output streams. There is also recursion with a method for walking a file tree that creates a FileVisitor interface implementation, either default or custom, with 3 phases: previsit directory, visit file, and postvisit directory. It also provides for a visit file failed method. At each point you can continue, stop, skip subtree, or skip other entries in the current directory. Options control whether symlinks are followed, and cycles are detected. DirectoryStream includes filters like glob, regex, and custom implementations that could be perfect for implementing Selectors. DirectoryStream uses less resources and works much better with remote resources. It also handles directories that are very large much more efficiently. Error handling throughout is by exception rather than boolean value as it is now, which I'm sure we all agree is preferable. Then there is the Watch Service which uses the native eventing service if available, where you can get events in your build when a file is created or deleted, or when a directory is modified. Would we translate it into something for BuildListeners? How would we harness it, scheduling a target and its dependencies when the a watch fired an event? I could keep going but this is getting rather long. I'll just point out a few resources from other people that are clearer than I am and have more detail: The Java 7 feature list[3] is comprehensive but has little on the impact of features; this Doctor Dobbs post[4] has good code examples for how NIO 2 could be used by Ant; an article listing "8 things I found most useful"[5], all of which are relevant to a simplified Ant implementation, details the NIO 2 changes at #8; interviews with Alan Bateman in text[6], video[7], and podcast[8] are interesting, as is a JavaOne 2011 slide show[9]; Leandro Pezzente sent me a link[10] (thanks, Leandro) that lists some of the other changes to Java 7 to think about, although it only mentions the file system changes that JSR 203 introduces in passing. [1] http://www.tbray.org/ongoing/When/200x/2008/05/01/Wide-Finder-2 [2] http://www.tbray.org/ongoing/When/200x/2007/10/30/WF-Results [3] http://openjdk.java.net/projects/jdk7/features/#f250 [4] http://drdobbs.com/blogs/jvm/231600403 [5] http://jaxenter.com/java-7-the-top-8-features-37156.html [6] http://www.artima.com/lejava/articles/more_new_io.html [7] http://w
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
On 2/16/2012 2:36 PM, Nicolas Lalevée wrote: Le 16 févr. 2012 à 20:47, Bruce Atherton a écrit : I'd hope to go further than that in backwards compatibility. I work with a lot of companies that are: a) resistant to learning new things unless there is a good reason for it (such as the migration from Apache HTTP Server from 1.x to 2.x to resolve security issues) b) have a number of separate Ant build scripts that follow different standards in different areas of the company, particularly if they have and c) need to have a justification to allocate resources to upgrade and change a working build to use new features, which standardizing builds across the organization using new features in a major release that simplify the build system may offer them. I don't know conclusion you're having there. Such companies shouldn't worry about any new major version, because they actually do want to stick with the old one for stability purpose. I guess that the companies which would be troubled is the ones which want to keep up with the releases, migrations from one version to another should not be too painful. No, you are right that these companies need a good reason to upgrade. What I am saying is that the pain that is caused by trying to make minor modifications to large complicated build systems, combined with having multiple large build systems that do very different things and the difficulty in dealing with these major changes in operating, is the thing that can cause at least some of them to dedicate the resources to standardize on something simpler. But I haven't found any that are interested in considering a new build system. They know Ant. Any upgrades they want to do in their own time. That is why I think backward compatibility is so important. They can roll out an upgrade to Ant 2, make sure everything works as expected, and then in their own time roll out a simplified, standardized build to each of the systems they are currently running it. Well, again, I think it's already there, no need to wait for an Ant 2.0 :) If you add the groovy-front.jar in Ant's boot classpath, write a build.groovy, then a launch of ant with no parameter on the command line will execute your groovy build script. See ProjectHelperRepository.getProjectHelperForBuildFile(Resource) I've got a lot of customers with the kinds of Ant build systems I am talking about. Precisely zero of them use anything other than an XML format for the build. Downloading extra bits to do funky things is not in their DNA. Some of them are forbidden to use Ant Contrib because it hasn't been through a security audit. I used to think we are living in a Maven world and that Ant was fine being just in maintenance mode. Since I've been helping these customers integrate my product with their systems, including their build systems, I've come to realize that there is still a very strong need for Ant out there, and that they are hurting from the complexity of it. Some of the complexity is from build systems that were written pre-macrodef and they haven't seen that one feature as compelling enough to commit to a rewrite. Some use macros, though, and things are still pretty complicated. I think this is Ant's market: places that don't want the dependency features of Maven and require complete control over exactly how their build is done. A lot of companies have their own, internally written build file generators just so their build systems are consistent and exactly what they want. Our Related Projects and External Tools page has some of these that were made public, I suspect. Surely there is a better way than requiring users of Ant to write generators to deal with the complexity and keep it customized. - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
Le 16 févr. 2012 à 20:49, Mansour Al Akeel a écrit : > 2012/2/16 Nicolas Lalevée > >> >> >> I cannot talk about Gradle because I never really understand the real >> motive apart from the apparent cool groovy language features. >> >> On the other hand, Easyant is about using Ant on steroïds. The idea is >> basically sharing Ant build scripts. >> Each time I have to make a build of a Java webapp, I don't write my >> build.xml from scratch each time. I look up for an old project I used to >> work on, I copy its build and keep the interesting parts, I rehack the >> build scripts. So with some convention and for very similar projects, we >> could share theses scripts. Easyant does "just" this with Ivy. >> > > I used gradle and don't see the reason for it except for the multi module > support. And yes, it's slow. For a continuous build like (on file change, > compile, build, and let jrebel reload), It's to slow for me. > > I understand that easyant makes reusable build available, but I think the > multi project support (in gradle), the groovy feature (in gradle), and the > reusable tasks (in easyant), can all be done with antlib. Why did their > teams created a new build system, rather than antlib ? Easyant doesn't per se provide reusable tasks, that is effectively the role of antlibs, but it better provides a standard build workflow, it provides targets. Extension points which makes target reusable were introduced in Ant after some prototyping in Easyant actually. I guess that a proper multi project support involves some build workflow management, so working with targets. I doubt it is feasible with just antlibs. Nicolas - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
Le 16 févr. 2012 à 21:08, Bruce Atherton a écrit :
> It has but not for quite a long time. Look in the archives from 2001 to 2003
> for "Mutant"[1] which Conor proposed, and "Myrmidon"[2] which Peter Donald
> proposed back in 2000. You can still find them in the svn repository[3], [4].
>
> I think there was so much discussion on a new design of Ant that everyone
> just got exhausted talking about it. As I recall what finally brought it to a
> halt was Costin Manolache saying "Just refactor what you have while retaining
> backward compatibility."
>
> The general agreements that I remember, although I haven't trawled the
> mailing list to find references, were that backward binary compatibility
> could only be broken through an Ant 2.0 release, and that Ant 2.0 should do
> everything in its power to be build file compatible. The thinking then was an
> XSLT file could be provided if necessary although at this point I think we
> could provide an task even if it just ran an XSLT, should
> that prove necessary. But I don't think it should be required if possible, at
> least not for several minor releases.
I agree, the build file should be maintained compatible. As I wrote earlier a
"version" tag should do the trick if new features need to break it a little
bit, no version tag would mean it is a <2.0 build file.
> This is a new group of Ant developers, though, and they may make different
> decisions than the ones back then did. If we find volunteers willing to step
> forward to help with the code. I can do the infrastructure things like
> setting up a place to put everything in subversion, perhaps in
> ant/sandbox/{some code name}. Any suggestions? All I can think of is pezant
> or something similarly punny. It could just be ant2proposal.
If it goes further, I will be happy to keep a eye around and discuss about it
but I won't be able to write any code.
Nicolas
-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
Le 16 févr. 2012 à 20:47, Bruce Atherton a écrit : > I'd hope to go further than that in backwards compatibility. I work with a > lot of companies that are: > >a) resistant to learning new things unless there is a good reason for it > (such as the migration from Apache HTTP Server from 1.x to 2.x to resolve > security issues) > >b) have a number of separate Ant build scripts that follow different > standards in different areas of the company, particularly if they have > > and c) need to have a justification to allocate resources to upgrade and > change a working build to use new features, which standardizing builds across > the organization using new features in a major release that simplify the > build system may offer them. I don't know conclusion you're having there. Such companies shouldn't worry about any new major version, because they actually do want to stick with the old one for stability purpose. I guess that the companies which would be troubled is the ones which want to keep up with the releases, migrations from one version to another should not be too painful. > You are right about the plugin parser architecture in Ant 1.x, but one of the > problems with it is that there is nothing shipped by default. What I meant > was that I would love it if Ant 2 also represented a point where new builds > could think about using a new build format automatically, just based on file > extension or a flag on the command line. That might encourage new projects to > adopt it. Well, again, I think it's already there, no need to wait for an Ant 2.0 :) If you add the groovy-front.jar in Ant's boot classpath, write a build.groovy, then a launch of ant with no parameter on the command line will execute your groovy build script. See ProjectHelperRepository.getProjectHelperForBuildFile(Resource) Nicolas - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
It has but not for quite a long time. Look in the archives from 2001 to
2003 for "Mutant"[1] which Conor proposed, and "Myrmidon"[2] which
Peter Donald proposed back in 2000. You can still find them in the svn
repository[3], [4].
I think there was so much discussion on a new design of Ant that
everyone just got exhausted talking about it. As I recall what finally
brought it to a halt was Costin Manolache saying "Just refactor what you
have while retaining backward compatibility."
The general agreements that I remember, although I haven't trawled the
mailing list to find references, were that backward binary compatibility
could only be broken through an Ant 2.0 release, and that Ant 2.0 should
do everything in its power to be build file compatible. The thinking
then was an XSLT file could be provided if necessary although at this
point I think we could provide an task even if it
just ran an XSLT, should that prove necessary. But I don't think it
should be required if possible, at least not for several minor releases.
This is a new group of Ant developers, though, and they may make
different decisions than the ones back then did. If we find volunteers
willing to step forward to help with the code. I can do the
infrastructure things like setting up a place to put everything in
subversion, perhaps in ant/sandbox/{some code name}. Any suggestions?
All I can think of is pezant or something similarly punny. It could just
be ant2proposal.
As for whether it is a runtime issue, as far as I know the only problem
with libraries is the bootstrap build. If Ant 2's bootstrap build is
done by Ant 1 there is no problem with adding any libraries you want to
the core of Ant 2. If it is self-hosted then Ant 2 needs to be capable
of running without any external libraries.
This is why the Ant 1.x codebase has a Java tar and zip and bzip2
implementation hosted within it, even though Apache Commons has superior
implementations of all of them. Ant source is distributed in these
formats so Ant needs a bootstrap way to get at it. Stefan maintains the
compress AntLib for exactly this reason, so Ant can have access to a
really good, full featured compression library.
[1] http://marc.info/?l=ant-dev&m=98934355711364&w=2
[2] http://marc.info/?l=ant-dev&m=97593652718829&w=2
[3]
http://svn.apache.org/viewvc/ant/core/tags/ANT_152_FINAL/proposal/mutant/
[4] http://svn.apache.org/viewvc/ant/core/tags/ANT_141/proposal/myrmidon/
On 2/13/2012 1:30 PM, Mansour Al Akeel wrote:
interesting info. It looks like the idea of the redesign has been discussed
a lot in the past.
Another good point, is to have ant independent of any external libraries.
However, I am wondering if this applies to run time environment ?
For example, writing a core ant (mainly build.xml parser), as an osgi
bundle. And collection of bundles for Javac, Java, Copy,... etc. would:
1- be independent of any external libraries and relies on JRE to build.
2- allow integration with IDEs.
3- allow to compile and build the build system, without a build system (ie,
using bootstrap). or like you said "self-building".
Would this be acceptable idea ? A core bundle, and extra bundles for basic
tasks. A bundle for ivy (maybe). We can even have a bundle to install
additional bundles remotely
And with Java7 NIO the performance will be fine.
comments ?
-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
On Thu, Feb 16, 2012 at 2:49 PM, Mansour Al Akeel wrote: > > > > 2012/2/16 Nicolas Lalevée > > > > implementation are, but a 1s launch (bash script and jvm launch included) > > would be to too long for me. > > > Agreed. > > And this is why I suggested java plugin framework in a previous email > (assuming it is faster than OSGI). I read that there are some fast OSGI > implementations. Didn't try anything yet. > I'm neither an OSGi fan or bigot, but in my experience OSGi itself, as a specification, does not imply poor performance. There are some aspects of the OSGi runtime package wiring specification that are NP-complete, but this typically doesn't hurt real performance unless you have a system of thousands of bundles like Eclipse. I would hope something like Ant never got that far. In particular, loading the actual OSGi runtime is very fast, on the order of milliseconds. Keep in mind that the original OSGi runtime environment was (and still is) embedded systems with limited memory and CPU. How the application is designed, such as the degree of modularity, is ultimately the biggest determinant of performance. To that end I don't know if we could compare an (as yet unknown) standard Java modularity system) to OSGi, but I would be surprised if the native system was significantly more performant--unless it gave up some of the invaluable features that OSGi provides. As an aside, and speaking of Eclipse, you would want to keep in mind how to embed an Ant containing an OSGi runtime inside Eclipse. This isn't impossible, but it could lead to some interesting classloader hierarchies. Rich
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
Oops, accidental deletion. On 2/16/2012 11:47 AM, Bruce Atherton wrote: b) have a number of separate Ant build scripts that follow different standards in different areas of the company, particularly if they have acquired other companies with their own Ant builds. - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
2012/2/16 Nicolas Lalevée > > > I cannot talk about Gradle because I never really understand the real > motive apart from the apparent cool groovy language features. > > On the other hand, Easyant is about using Ant on steroïds. The idea is > basically sharing Ant build scripts. > Each time I have to make a build of a Java webapp, I don't write my > build.xml from scratch each time. I look up for an old project I used to > work on, I copy its build and keep the interesting parts, I rehack the > build scripts. So with some convention and for very similar projects, we > could share theses scripts. Easyant does "just" this with Ivy. > I used gradle and don't see the reason for it except for the multi module support. And yes, it's slow. For a continuous build like (on file change, compile, build, and let jrebel reload), It's to slow for me. I understand that easyant makes reusable build available, but I think the multi project support (in gradle), the groovy feature (in gradle), and the reusable tasks (in easyant), can all be done with antlib. Why did their teams created a new build system, rather than antlib ? This is my question. > > > Is there something wrong with antlib ? Would OSGI be more convenient and > > appealing for programmers to create and contribute their plugins rather > > than writing their own build system ? Derivatives of eclipse exists, but > > mainly they are just bundles of different plugins. Is this because it's > > OSGI ? > > I like this idea of using of modular framework to handle properly > modularity. > But what I wouldn't like is having a longer bootstrapping of Ant. For > instance gradle is boring long to launch, even for printing the inline > help. The groovy frontend of Ant suffers for the same issues, unusable for > me. I don't have much experience with OSGi systems, I don't know how fast > some > > 2012/2/16 Nicolas Lalevée > > implementation are, but a 1s launch (bash script and jvm launch included) > would be to too long for me. > > And this is why I suggested java plugin framework in a previous email (assuming it is faster than OSGI). I read that there are some fast OSGI implementations. Didn't try anything yet. > Nicolas > > > - > To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org > For additional commands, e-mail: dev-h...@ant.apache.org > >
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
I'd hope to go further than that in backwards compatibility. I work with a lot of companies that are: a) resistant to learning new things unless there is a good reason for it (such as the migration from Apache HTTP Server from 1.x to 2.x to resolve security issues) b) have a number of separate Ant build scripts that follow different standards in different areas of the company, particularly if they have and c) need to have a justification to allocate resources to upgrade and change a working build to use new features, which standardizing builds across the organization using new features in a major release that simplify the build system may offer them. You are right about the plugin parser architecture in Ant 1.x, but one of the problems with it is that there is nothing shipped by default. What I meant was that I would love it if Ant 2 also represented a point where new builds could think about using a new build format automatically, just based on file extension or a flag on the command line. That might encourage new projects to adopt it. On 2/16/2012 10:57 AM, Nicolas Lalevée wrote: Le 14 févr. 2012 à 20:02, Bruce Atherton a écrit : On 2/14/2012 6:13 AM, Stefan Bodewig wrote: This will lead us to the discussion of what Ant2 would be. A rewritten Ant that remains compatible (or mostly so) on the build file level or something quite different? My opinion. I think we need at least an option for being backwards compatible at the build file level and it should be the default for quite a while after an initial 2.0 release. A simple trick can be a declaration of a "version" attribute which would declare the minimum Ant version required to run the build file, and also declare how it should be interpreted. This is how it works for the ivy.xml for instance. I'd hope, though, that a redesign would have a fully pluggable parser so that even with the first release there was an option to use something other than XML. I think you can already today, there are some experiments which show it works [1] [2]. The names of the classes of Ant may not be explicit, but I think the abstraction is there. Nicolas [1] http://svn.apache.org/repos/asf/ant/sandbox/javafront/ [2] http://svn.apache.org/repos/asf/ant/sandbox/groovyfront/ - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
Le 15 févr. 2012 à 20:05, Mansour Al Akeel a écrit : > Another thing I don't understand about the current Ant. Why there are > derivatives from ant and they are gaining popularity ? I am talking about > gradle and easyant. > > Gradle adds mutli project support, and easyant sets some conventions (I > didn't use it). I am wondering what makes the author go with a new build > system, if things like this can be done with antlib. And why would users > switch to these systems. I like writing my custom tasks and wrap them in an > antlib. But why others are not doing the same to add multi project support, > or set some conventions ? I cannot talk about Gradle because I never really understand the real motive apart from the apparent cool groovy language features. On the other hand, Easyant is about using Ant on steroïds. The idea is basically sharing Ant build scripts. Each time I have to make a build of a Java webapp, I don't write my build.xml from scratch each time. I look up for an old project I used to work on, I copy its build and keep the interesting parts, I rehack the build scripts. So with some convention and for very similar projects, we could share theses scripts. Easyant does "just" this with Ivy. > Is there something wrong with antlib ? Would OSGI be more convenient and > appealing for programmers to create and contribute their plugins rather > than writing their own build system ? Derivatives of eclipse exists, but > mainly they are just bundles of different plugins. Is this because it's > OSGI ? I like this idea of using of modular framework to handle properly modularity. But what I wouldn't like is having a longer bootstrapping of Ant. For instance gradle is boring long to launch, even for printing the inline help. The groovy frontend of Ant suffers for the same issues, unusable for me. I don't have much experience with OSGi systems, I don't know how fast some implementation are, but a 1s launch (bash script and jvm launch included) would be to too long for me. Nicolas - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
Le 14 févr. 2012 à 20:02, Bruce Atherton a écrit : > On 2/14/2012 6:13 AM, Stefan Bodewig wrote: >> >> This will lead us to the discussion of what Ant2 would be. A rewritten >> Ant that remains compatible (or mostly so) on the build file level or >> something quite different? >> > > My opinion. > > I think we need at least an option for being backwards compatible at the > build file level and it should be the default for quite a while after an > initial 2.0 release. A simple trick can be a declaration of a "version" attribute which would declare the minimum Ant version required to run the build file, and also declare how it should be interpreted. This is how it works for the ivy.xml for instance. > I'd hope, though, that a redesign would have a fully pluggable parser so that > even with the first release there was an option to use something other than > XML. I think you can already today, there are some experiments which show it works [1] [2]. The names of the classes of Ant may not be explicit, but I think the abstraction is there. Nicolas [1] http://svn.apache.org/repos/asf/ant/sandbox/javafront/ [2] http://svn.apache.org/repos/asf/ant/sandbox/groovyfront/ - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: NIO 2.0 == Ant 2.0? was Re: Java NIO support
On 2/15/2012 5:48 PM, Bruce Atherton wrote: I've read a blog post that said that one of the requirements that has been adopted is OSGI compatibility... Here is the post I mentioned: http://osgithoughts.blogspot.com/2011/05/java-se-8-modularity-requirements.html It brings up a good point: a system architected with modularity in mind is fundamentally different from a system that isn't. Ant 1.x is modular, but there are many different kinds of modularity introduced over time and it can be hard to keep track of which files need to be modified to add a particular build file element. Some are pluggable, some load from a properties list, etc. Sometimes, particularly for backward API compatibility, you need to add a new element to more than one place and it can be easy to miss one. Having a single module system for the whole system that is compatible with what is expected in Java 8, with the option of plugging OSGI in to replace it, seems like a good solution to me. When Java 8 is the lowest supported JVM of Ant 2 (sometime 5 years from now based on previous experience) we can drop our solution and use whatever is bundled as the default. - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
