Re: Easyant - Plugin conflict management
My answers below 2013/8/26 Dridi Boukelmoune dridi.boukelmo...@zenika.com Point 2 : Even if there is no much activity, i suggest to keep backward compatibility fine for me. Not necessary from my point of view. My only advice is to avoid legacy (or technical debt ;) at such early time. I was suggesting to keep backward compatibility mainly because easyant is build by easyant :). Though we need older plugins to build the new easyant and vice et versa. Point 3 : By two steps i meant running a global resolve (for all plugins and buildtypes including transitive ones). And then have a second class invoked to perform individual import of ant build file by picking them from the ResolveReport. ok I see. This make totally sense. This is where I'm lost. Actually the whole issue of dependencies conflicts between plugins. Here is the behavior I think I've observed with Maven: - maven resolves project dependencies and does a shitty job at conflicts resolution - maven resolves plugins dependencies one by one and downloads them lazily - each plugin is executed with its own classloader, and doesn't care about other plugins dependencies - I can get a dozen versions of the infamous[1] plexus-utils in a single build My point is, given proper isolation, is it a real issue to have different plugins depending on different versions of the same modules ? The main point is to avoid having dozen version of jars :). But we also have a internal limitation, ant doesn't load a buildfile two times. So if we have many plugins relying on abstract ones, abstract ones must be uniquely resolved. -- Jean Louis Boudart Independent consultant Apache EasyAnt commiter http://ant.apache.org/easyant/
Re: Easyant - Plugin conflict management
On Wed, Aug 28, 2013 at 11:19 AM, Jean-Louis Boudart jeanlouis.boud...@gmail.com wrote: My answers below 2013/8/26 Dridi Boukelmoune dridi.boukelmo...@zenika.com Point 2 : Even if there is no much activity, i suggest to keep backward compatibility fine for me. Not necessary from my point of view. My only advice is to avoid legacy (or technical debt ;) at such early time. I was suggesting to keep backward compatibility mainly because easyant is build by easyant :). Though we need older plugins to build the new easyant and vice et versa. Forgot about that. Point 3 : By two steps i meant running a global resolve (for all plugins and buildtypes including transitive ones). And then have a second class invoked to perform individual import of ant build file by picking them from the ResolveReport. ok I see. This make totally sense. This is where I'm lost. Actually the whole issue of dependencies conflicts between plugins. Here is the behavior I think I've observed with Maven: - maven resolves project dependencies and does a shitty job at conflicts resolution - maven resolves plugins dependencies one by one and downloads them lazily - each plugin is executed with its own classloader, and doesn't care about other plugins dependencies - I can get a dozen versions of the infamous[1] plexus-utils in a single build My point is, given proper isolation, is it a real issue to have different plugins depending on different versions of the same modules ? The main point is to avoid having dozen version of jars :). I agree on the fact this can easily become a classloader hell, but my concern is the dependency hell. Again, I have no answer to your question, I'm just asking more questions :) But we also have a internal limitation, ant doesn't load a buildfile two times. So if we have many plugins relying on abstract ones, abstract ones must be uniquely resolved. I think I lack context to understand this one. -- Jean Louis Boudart Independent consultant Apache EasyAnt commiter http://ant.apache.org/easyant/ Best Regards, Dridi - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Easyant - Plugin conflict management
Inhalt uebernehmen. Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité pour le contenu fourni. From: dridi.boukelmo...@zenika.com Date: Mon, 26 Aug 2013 20:31:32 +0200 Subject: Re: Easyant - Plugin conflict management To: dev@ant.apache.org Hi, This is my first post on this mailing list, I used to lurk in the previous easant ML and even answer sometimes. In this thread I see several points and one of them looks like a non-issue to me. Please forgive my heresy when I use maven as an example. On Tue, Aug 20, 2013 at 6:50 PM, Nicolas Lalevée nicolas.lale...@hibnet.org wrote: Le 20 août 2013 à 14:22, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : I've commited changes on trunk with some unit tests if you want to have a look. Known limitation : * system plugins doesn't participate *yet* to global plugin resolution * plugins classpath created dinamically will all contains the whole resolution graph (not really a problem) Point 1 : this can even affect performance. Invoking resolve process one time seems much more speed that invoking plugins individually. This seems strange to me. Do you have an idea why ? I believe he meant this could *improve* performances by sharing a single global resolve between all plugins (or something like that). Point 2 : Even if there is no much activity, i suggest to keep backward compatibility fine for me. Not necessary from my point of view. My only advice is to avoid legacy (or technical debt ;) at such early time. Point 3 : By two steps i meant running a global resolve (for all plugins and buildtypes including transitive ones). And then have a second class invoked to perform individual import of ant build file by picking them from the ResolveReport. ok I see. This make totally sense. This is where I'm lost. Actually the whole issue of dependencies conflicts between plugins. Here is the behavior I think I've observed with Maven: - maven resolves project dependencies and does a shitty job at conflicts resolution - maven resolves plugins dependencies one by one and downloads them lazily - each plugin is executed with its own classloader, and doesn't care about other plugins dependencies - I can get a dozen versions of the infamous[1] plexus-utils in a single build My point is, given proper isolation, is it a real issue to have different plugins depending on different versions of the same modules ? At the very least, I like the idea of a single global resolve. Even though I understand the benefits of a lazy resolution/download, I don't see this as a problem with today's average bandwidth and disk space to get every thing needed by a build in a single go. I tend to work offline and I hate when I discover in the train that I'm missing a dependency :) You can check ResolvePlugins and ImportDeferred class in trunk if you want to see concrete stuff. I usually do a very quick review of the code to keep me informed of what is going on and see if nothing bug me, and as always, it seems great. But I need to get into this more deeply to see how it really works. Nicolas Best Regards, Dridi [1] it's just that I don't like it :p 2013/8/17 Nicolas Lalevée nicolas.lale...@hibnet.org Long overdue response. Le 3 août 2013 à 13:33, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : Hi there, It becomes necesssary to manage conflicts between plugins. This issues has been raised many time and is referenced on jira[1]. Currently easyant offers import task taking some specific attributes to resolve a plugin (mainly organisation, module name and revision). ea:import org=mycompany module=myplugin revision=x.x/ This task will : * resolve the given plugin or buildtypr * create a dynamic classpath for this plugin * expose location of others extra files through properties (ex checkstyle.xml containing checkstyle rules, this files is shipped in the plugin). Thoses properties will then be reused by the plugin itself * import the real ant file (invoke the importTask from ant core under the hood) This task is currently used : * dynamically by easyant to load system plugins (skeletons for example) * dynamically by easyant when you specify ea:build or ea:plugin tags in module.ivy files * invoked in plugin ant file itselfs * invoked in module.ant if users has complex needs Additionnal there is two alliases for this task to import plugins and buildtype. ea:plugin module
Re: Easyant - Plugin conflict management
-xhtml:jar:1.0 [DEBUG] Included: org.apache.maven.doxia:doxia-core:jar:1.0 [DEBUG] Included: org.apache.maven.doxia:doxia-sink-api:jar:1.0 [DEBUG] Included: org.apache.maven.doxia:doxia-site-renderer:jar:1.0 [DEBUG] Included: org.codehaus.plexus:plexus-velocity:jar:1.1.7 [DEBUG] Included: org.apache.velocity:velocity:jar:1.5 [DEBUG] Included: commons-lang:commons-lang:jar:2.1 [DEBUG] Included: oro:oro:jar:2.0.8 [DEBUG] Included: commons-collections:commons-collections:jar:3.2 [DEBUG] Included: org.apache.maven.doxia:doxia-module-apt:jar:1.0 [DEBUG] Included: org.apache.maven.doxia:doxia-module-fml:jar:1.0 [DEBUG] Included: org.apache.maven.doxia:doxia-module-xdoc:jar:1.0 [DEBUG] Included: org.apache.maven.reporting:maven-reporting-api:jar:2.0.6 [DEBUG] Included: org.codehaus.plexus:plexus-i18n:jar:1.0-beta-6 [DEBUG] Included: org.codehaus.plexus:plexus-utils:jar:3.0.7 [DEBUG] Configuring mojo org.codehaus.mojo:clirr-maven-plugin:2.5:check from plugin realm ClassRealm[pluginorg.codehaus.mojo:clirr-maven-plugin:2.5, parent: sun.misc.Launcher$AppClassLoader@e1641c0] I'm omitting logs starting with Excluded (no plexus-utils in those). In a single build, I get three plugins using three different versions of plexus-utils (2.0.5, 3.0, 3.0.7). So this is what I meant: is it a problem for plugins to depend on different versions of other stuff (plugins or modules) ? http://maven.apache.org/pom.html#Plugin_Management Martin Gainty I hope I've convinced you I'm not freely flaming on Maven but really trying to contribute to the thread. Best Regards, Dridi __ Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopife ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen. Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité pour le contenu fourni. From: dridi.boukelmo...@zenika.com Date: Mon, 26 Aug 2013 20:31:32 +0200 Subject: Re: Easyant - Plugin conflict management To: dev@ant.apache.org Hi, This is my first post on this mailing list, I used to lurk in the previous easant ML and even answer sometimes. In this thread I see several points and one of them looks like a non-issue to me. Please forgive my heresy when I use maven as an example. On Tue, Aug 20, 2013 at 6:50 PM, Nicolas Lalevée nicolas.lale...@hibnet.org wrote: Le 20 août 2013 à 14:22, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : I've commited changes on trunk with some unit tests if you want to have a look. Known limitation : * system plugins doesn't participate *yet* to global plugin resolution * plugins classpath created dinamically will all contains the whole resolution graph (not really a problem) Point 1 : this can even affect performance. Invoking resolve process one time seems much more speed that invoking plugins individually. This seems strange to me. Do you have an idea why ? I believe he meant this could *improve* performances by sharing a single global resolve between all plugins (or something like that). Point 2 : Even if there is no much activity, i suggest to keep backward compatibility fine for me. Not necessary from my point of view. My only advice is to avoid legacy (or technical debt ;) at such early time. Point 3 : By two steps i meant running a global resolve (for all plugins and buildtypes including transitive ones). And then have a second class invoked to perform individual import of ant build file by picking them from the ResolveReport. ok I see. This make totally sense. This is where I'm lost. Actually the whole issue of dependencies conflicts between plugins. Here is the behavior I think I've observed with Maven: - maven resolves project dependencies and does a shitty job at conflicts resolution - maven resolves plugins dependencies one by one and downloads them lazily - each plugin is executed with its own classloader, and doesn't care about other plugins dependencies - I can get a dozen versions of the infamous[1] plexus-utils in a single build My point is, given proper isolation, is it a real issue to have different
Re: Easyant - Plugin conflict management
Hi, This is my first post on this mailing list, I used to lurk in the previous easant ML and even answer sometimes. In this thread I see several points and one of them looks like a non-issue to me. Please forgive my heresy when I use maven as an example. On Tue, Aug 20, 2013 at 6:50 PM, Nicolas Lalevée nicolas.lale...@hibnet.org wrote: Le 20 août 2013 à 14:22, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : I've commited changes on trunk with some unit tests if you want to have a look. Known limitation : * system plugins doesn't participate *yet* to global plugin resolution * plugins classpath created dinamically will all contains the whole resolution graph (not really a problem) Point 1 : this can even affect performance. Invoking resolve process one time seems much more speed that invoking plugins individually. This seems strange to me. Do you have an idea why ? I believe he meant this could *improve* performances by sharing a single global resolve between all plugins (or something like that). Point 2 : Even if there is no much activity, i suggest to keep backward compatibility fine for me. Not necessary from my point of view. My only advice is to avoid legacy (or technical debt ;) at such early time. Point 3 : By two steps i meant running a global resolve (for all plugins and buildtypes including transitive ones). And then have a second class invoked to perform individual import of ant build file by picking them from the ResolveReport. ok I see. This make totally sense. This is where I'm lost. Actually the whole issue of dependencies conflicts between plugins. Here is the behavior I think I've observed with Maven: - maven resolves project dependencies and does a shitty job at conflicts resolution - maven resolves plugins dependencies one by one and downloads them lazily - each plugin is executed with its own classloader, and doesn't care about other plugins dependencies - I can get a dozen versions of the infamous[1] plexus-utils in a single build My point is, given proper isolation, is it a real issue to have different plugins depending on different versions of the same modules ? At the very least, I like the idea of a single global resolve. Even though I understand the benefits of a lazy resolution/download, I don't see this as a problem with today's average bandwidth and disk space to get every thing needed by a build in a single go. I tend to work offline and I hate when I discover in the train that I'm missing a dependency :) You can check ResolvePlugins and ImportDeferred class in trunk if you want to see concrete stuff. I usually do a very quick review of the code to keep me informed of what is going on and see if nothing bug me, and as always, it seems great. But I need to get into this more deeply to see how it really works. Nicolas Best Regards, Dridi [1] it's just that I don't like it :p 2013/8/17 Nicolas Lalevée nicolas.lale...@hibnet.org Long overdue response. Le 3 août 2013 à 13:33, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : Hi there, It becomes necesssary to manage conflicts between plugins. This issues has been raised many time and is referenced on jira[1]. Currently easyant offers import task taking some specific attributes to resolve a plugin (mainly organisation, module name and revision). ea:import org=mycompany module=myplugin revision=x.x/ This task will : * resolve the given plugin or buildtypr * create a dynamic classpath for this plugin * expose location of others extra files through properties (ex checkstyle.xml containing checkstyle rules, this files is shipped in the plugin). Thoses properties will then be reused by the plugin itself * import the real ant file (invoke the importTask from ant core under the hood) This task is currently used : * dynamically by easyant to load system plugins (skeletons for example) * dynamically by easyant when you specify ea:build or ea:plugin tags in module.ivy files * invoked in plugin ant file itselfs * invoked in module.ant if users has complex needs Additionnal there is two alliases for this task to import plugins and buildtype. ea:plugin module=compile-java revision=0.9/ ea:plugin module=build-std-java revision=0.9/ If organisation attribute is not specified on aliases default one will be used. It does the job but it doesn't handle conflict between plugins. Some plugins relies on abstract ones. Exemple: package-jar depends on abstract-package, abstract-package depends on abstract-compile, but compile-java plugin also depends on abstract-compile. Which versi of abstract-compile will be taken in case both plugins load different version ? The answer is the first one ! This becomes more problematic on buildtypes, as buildtypes loads a set of plugins (including themself others abstract-plugins). Ok so now you should have a quick picture of the problem. What could be done ? We can rely on ivy to describe
RE: Easyant - Plugin conflict management
you never looked at plugin-management understanding what you're criticising before you write a commentary will lend veracity to your statements http://maven.apache.org/pom.html#Plugin_Management Martin Gainty __ Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopife ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen. Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité pour le contenu fourni. From: dridi.boukelmo...@zenika.com Date: Mon, 26 Aug 2013 20:31:32 +0200 Subject: Re: Easyant - Plugin conflict management To: dev@ant.apache.org Hi, This is my first post on this mailing list, I used to lurk in the previous easant ML and even answer sometimes. In this thread I see several points and one of them looks like a non-issue to me. Please forgive my heresy when I use maven as an example. On Tue, Aug 20, 2013 at 6:50 PM, Nicolas Lalevée nicolas.lale...@hibnet.org wrote: Le 20 août 2013 à 14:22, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : I've commited changes on trunk with some unit tests if you want to have a look. Known limitation : * system plugins doesn't participate *yet* to global plugin resolution * plugins classpath created dinamically will all contains the whole resolution graph (not really a problem) Point 1 : this can even affect performance. Invoking resolve process one time seems much more speed that invoking plugins individually. This seems strange to me. Do you have an idea why ? I believe he meant this could *improve* performances by sharing a single global resolve between all plugins (or something like that). Point 2 : Even if there is no much activity, i suggest to keep backward compatibility fine for me. Not necessary from my point of view. My only advice is to avoid legacy (or technical debt ;) at such early time. Point 3 : By two steps i meant running a global resolve (for all plugins and buildtypes including transitive ones). And then have a second class invoked to perform individual import of ant build file by picking them from the ResolveReport. ok I see. This make totally sense. This is where I'm lost. Actually the whole issue of dependencies conflicts between plugins. Here is the behavior I think I've observed with Maven: - maven resolves project dependencies and does a shitty job at conflicts resolution - maven resolves plugins dependencies one by one and downloads them lazily - each plugin is executed with its own classloader, and doesn't care about other plugins dependencies - I can get a dozen versions of the infamous[1] plexus-utils in a single build My point is, given proper isolation, is it a real issue to have different plugins depending on different versions of the same modules ? At the very least, I like the idea of a single global resolve. Even though I understand the benefits of a lazy resolution/download, I don't see this as a problem with today's average bandwidth and disk space to get every thing needed by a build in a single go. I tend to work offline and I hate when I discover in the train that I'm missing a dependency :) You can check ResolvePlugins and ImportDeferred class in trunk if you want to see concrete stuff. I usually do a very quick review of the code to keep me informed of what is going on and see if nothing bug me, and as always, it seems great. But I need to get into this more deeply to see how it really works. Nicolas Best Regards, Dridi [1] it's just that I don't like it :p 2013/8/17 Nicolas Lalevée nicolas.lale...@hibnet.org Long overdue response. Le 3 août 2013 à 13:33, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : Hi there, It becomes necesssary to manage conflicts between plugins. This issues has been raised many time and is referenced on jira[1]. Currently easyant offers import task taking some specific attributes to resolve a plugin (mainly organisation, module name and revision). ea:import org=mycompany module=myplugin revision=x.x/ This task will : * resolve the given plugin
Re: Easyant - Plugin conflict management
I've commited changes on trunk with some unit tests if you want to have a look. Known limitation : * system plugins doesn't participate *yet* to global plugin resolution * plugins classpath created dinamically will all contains the whole resolution graph (not really a problem) Point 1 : this can even affect performance. Invoking resolve process one time seems much more speed that invoking plugins individually. Point 2 : Even if there is no much activity, i suggest to keep backward compatibility Point 3 : By two steps i meant running a global resolve (for all plugins and buildtypes including transitive ones). And then have a second class invoked to perform individual import of ant build file by picking them from the ResolveReport. You can check ResolvePlugins and ImportDeferred class in trunk if you want to see concrete stuff. 2013/8/17 Nicolas Lalevée nicolas.lale...@hibnet.org Long overdue response. Le 3 août 2013 à 13:33, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : Hi there, It becomes necesssary to manage conflicts between plugins. This issues has been raised many time and is referenced on jira[1]. Currently easyant offers import task taking some specific attributes to resolve a plugin (mainly organisation, module name and revision). ea:import org=mycompany module=myplugin revision=x.x/ This task will : * resolve the given plugin or buildtypr * create a dynamic classpath for this plugin * expose location of others extra files through properties (ex checkstyle.xml containing checkstyle rules, this files is shipped in the plugin). Thoses properties will then be reused by the plugin itself * import the real ant file (invoke the importTask from ant core under the hood) This task is currently used : * dynamically by easyant to load system plugins (skeletons for example) * dynamically by easyant when you specify ea:build or ea:plugin tags in module.ivy files * invoked in plugin ant file itselfs * invoked in module.ant if users has complex needs Additionnal there is two alliases for this task to import plugins and buildtype. ea:plugin module=compile-java revision=0.9/ ea:plugin module=build-std-java revision=0.9/ If organisation attribute is not specified on aliases default one will be used. It does the job but it doesn't handle conflict between plugins. Some plugins relies on abstract ones. Exemple: package-jar depends on abstract-package, abstract-package depends on abstract-compile, but compile-java plugin also depends on abstract-compile. Which versi of abstract-compile will be taken in case both plugins load different version ? The answer is the first one ! This becomes more problematic on buildtypes, as buildtypes loads a set of plugins (including themself others abstract-plugins). Ok so now you should have a quick picture of the problem. What could be done ? We can rely on ivy to describe dependency on plugins. But then how could we know in which order plugins should be loaded ? I suggest to introduce a deferred import mechanism. We should split responsibility in two distinct steps. 1 - resolve (based on ivy) the whole graph of plugins and store the resolve report somewhere as a reusable reference in ant project 2 - invoke a new import task should import already resolved plugins (the task could rely on the report stored as a reference in ant project) Exemple : compile java will have an ivy dependency on abstract-compile dependency org=org.apache.easyant.plugins module=abstract-compile revision=1.0/ The compile java ant script will import the resolved plugin ea:import-deferred org=org.apache.easyant.plugins module=abstract-compile/ Note that i'm not fixed yet with the task name. Any suggestion (even for alliases are welcome). To maintain backward compatibility i'm in favor of creating new aliases import-plugin and import-buildtype instead of reusing existing ones. People would be able to continue using existing task with known the limitation (no conflict management on plugins). This can help if someone wants to load plugins in module.ant after setting a few properties. I also recommend adding a warning on existing task to recommend people using the new import mechanism. What do you think ? I see 3 points here. First, managing plugin dependencies: with Ivy, of course, we couldn't agree more :) Then about creating new tasks to keep backward compatibility. I think we can break backward compatibility. Easyant is not yet 1.0 and I do not see much activity on the user list. I would prefer bugging the current users than having an error-prone and deprecated task around. Third there is the resolve in two steps. I really like the idea. I am not sure though if we need this in order to bring conflict management in plugin dependencies. And I am not sure how far you are willing to go. Actually this is a
Re: Easyant - Plugin conflict management
Le 20 août 2013 à 14:22, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : I've commited changes on trunk with some unit tests if you want to have a look. Known limitation : * system plugins doesn't participate *yet* to global plugin resolution * plugins classpath created dinamically will all contains the whole resolution graph (not really a problem) Point 1 : this can even affect performance. Invoking resolve process one time seems much more speed that invoking plugins individually. This seems strange to me. Do you have an idea why ? Point 2 : Even if there is no much activity, i suggest to keep backward compatibility fine for me. Point 3 : By two steps i meant running a global resolve (for all plugins and buildtypes including transitive ones). And then have a second class invoked to perform individual import of ant build file by picking them from the ResolveReport. ok I see. This make totally sense. You can check ResolvePlugins and ImportDeferred class in trunk if you want to see concrete stuff. I usually do a very quick review of the code to keep me informed of what is going on and see if nothing bug me, and as always, it seems great. But I need to get into this more deeply to see how it really works. Nicolas 2013/8/17 Nicolas Lalevée nicolas.lale...@hibnet.org Long overdue response. Le 3 août 2013 à 13:33, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : Hi there, It becomes necesssary to manage conflicts between plugins. This issues has been raised many time and is referenced on jira[1]. Currently easyant offers import task taking some specific attributes to resolve a plugin (mainly organisation, module name and revision). ea:import org=mycompany module=myplugin revision=x.x/ This task will : * resolve the given plugin or buildtypr * create a dynamic classpath for this plugin * expose location of others extra files through properties (ex checkstyle.xml containing checkstyle rules, this files is shipped in the plugin). Thoses properties will then be reused by the plugin itself * import the real ant file (invoke the importTask from ant core under the hood) This task is currently used : * dynamically by easyant to load system plugins (skeletons for example) * dynamically by easyant when you specify ea:build or ea:plugin tags in module.ivy files * invoked in plugin ant file itselfs * invoked in module.ant if users has complex needs Additionnal there is two alliases for this task to import plugins and buildtype. ea:plugin module=compile-java revision=0.9/ ea:plugin module=build-std-java revision=0.9/ If organisation attribute is not specified on aliases default one will be used. It does the job but it doesn't handle conflict between plugins. Some plugins relies on abstract ones. Exemple: package-jar depends on abstract-package, abstract-package depends on abstract-compile, but compile-java plugin also depends on abstract-compile. Which versi of abstract-compile will be taken in case both plugins load different version ? The answer is the first one ! This becomes more problematic on buildtypes, as buildtypes loads a set of plugins (including themself others abstract-plugins). Ok so now you should have a quick picture of the problem. What could be done ? We can rely on ivy to describe dependency on plugins. But then how could we know in which order plugins should be loaded ? I suggest to introduce a deferred import mechanism. We should split responsibility in two distinct steps. 1 - resolve (based on ivy) the whole graph of plugins and store the resolve report somewhere as a reusable reference in ant project 2 - invoke a new import task should import already resolved plugins (the task could rely on the report stored as a reference in ant project) Exemple : compile java will have an ivy dependency on abstract-compile dependency org=org.apache.easyant.plugins module=abstract-compile revision=1.0/ The compile java ant script will import the resolved plugin ea:import-deferred org=org.apache.easyant.plugins module=abstract-compile/ Note that i'm not fixed yet with the task name. Any suggestion (even for alliases are welcome). To maintain backward compatibility i'm in favor of creating new aliases import-plugin and import-buildtype instead of reusing existing ones. People would be able to continue using existing task with known the limitation (no conflict management on plugins). This can help if someone wants to load plugins in module.ant after setting a few properties. I also recommend adding a warning on existing task to recommend people using the new import mechanism. What do you think ? I see 3 points here. First, managing plugin dependencies: with Ivy, of course, we couldn't agree more :) Then about creating new tasks to keep backward compatibility. I think we can break backward compatibility. Easyant is not yet 1.0 and I do
Re: Easyant - Plugin conflict management
Long overdue response. Le 3 août 2013 à 13:33, Jean-Louis Boudart jeanlouis.boud...@gmail.com a écrit : Hi there, It becomes necesssary to manage conflicts between plugins. This issues has been raised many time and is referenced on jira[1]. Currently easyant offers import task taking some specific attributes to resolve a plugin (mainly organisation, module name and revision). ea:import org=mycompany module=myplugin revision=x.x/ This task will : * resolve the given plugin or buildtypr * create a dynamic classpath for this plugin * expose location of others extra files through properties (ex checkstyle.xml containing checkstyle rules, this files is shipped in the plugin). Thoses properties will then be reused by the plugin itself * import the real ant file (invoke the importTask from ant core under the hood) This task is currently used : * dynamically by easyant to load system plugins (skeletons for example) * dynamically by easyant when you specify ea:build or ea:plugin tags in module.ivy files * invoked in plugin ant file itselfs * invoked in module.ant if users has complex needs Additionnal there is two alliases for this task to import plugins and buildtype. ea:plugin module=compile-java revision=0.9/ ea:plugin module=build-std-java revision=0.9/ If organisation attribute is not specified on aliases default one will be used. It does the job but it doesn't handle conflict between plugins. Some plugins relies on abstract ones. Exemple: package-jar depends on abstract-package, abstract-package depends on abstract-compile, but compile-java plugin also depends on abstract-compile. Which versi of abstract-compile will be taken in case both plugins load different version ? The answer is the first one ! This becomes more problematic on buildtypes, as buildtypes loads a set of plugins (including themself others abstract-plugins). Ok so now you should have a quick picture of the problem. What could be done ? We can rely on ivy to describe dependency on plugins. But then how could we know in which order plugins should be loaded ? I suggest to introduce a deferred import mechanism. We should split responsibility in two distinct steps. 1 - resolve (based on ivy) the whole graph of plugins and store the resolve report somewhere as a reusable reference in ant project 2 - invoke a new import task should import already resolved plugins (the task could rely on the report stored as a reference in ant project) Exemple : compile java will have an ivy dependency on abstract-compile dependency org=org.apache.easyant.plugins module=abstract-compile revision=1.0/ The compile java ant script will import the resolved plugin ea:import-deferred org=org.apache.easyant.plugins module=abstract-compile/ Note that i'm not fixed yet with the task name. Any suggestion (even for alliases are welcome). To maintain backward compatibility i'm in favor of creating new aliases import-plugin and import-buildtype instead of reusing existing ones. People would be able to continue using existing task with known the limitation (no conflict management on plugins). This can help if someone wants to load plugins in module.ant after setting a few properties. I also recommend adding a warning on existing task to recommend people using the new import mechanism. What do you think ? I see 3 points here. First, managing plugin dependencies: with Ivy, of course, we couldn't agree more :) Then about creating new tasks to keep backward compatibility. I think we can break backward compatibility. Easyant is not yet 1.0 and I do not see much activity on the user list. I would prefer bugging the current users than having an error-prone and deprecated task around. Third there is the resolve in two steps. I really like the idea. I am not sure though if we need this in order to bring conflict management in plugin dependencies. And I am not sure how far you are willing to go. Actually this is a larger topic which has bugging me recently. The way we use the ivy.xml is generally either to tight or insecure. For instance when using version ranges in an ivy.xml, since the content of the repository is expected to change over time, then the resolve may change over time since new versions might fit the range. Range makes things unreliable over time, so often I restrict myself to not use any range. But it's kind of a shame to not use ranges in a dependency manager. I continued experimenting with the OSGi mapping in Ivy. And the OSGi version semantics are very loose. It is because it represents what versions of software it is compatible with, not the versions we will be using in your specific unit test or application. So relying on OSGi manifest to resolve dependencies is not safe at all. That's why I implemented the fixdeps [1] task. From a very loose specification of the dependencies, it will produce an ivy.xml which is very tight and secure. Then,
Easyant - Plugin conflict management
Hi there, It becomes necesssary to manage conflicts between plugins. This issues has been raised many time and is referenced on jira[1]. Currently easyant offers import task taking some specific attributes to resolve a plugin (mainly organisation, module name and revision). ea:import org=mycompany module=myplugin revision=x.x/ This task will : * resolve the given plugin or buildtypr * create a dynamic classpath for this plugin * expose location of others extra files through properties (ex checkstyle.xml containing checkstyle rules, this files is shipped in the plugin). Thoses properties will then be reused by the plugin itself * import the real ant file (invoke the importTask from ant core under the hood) This task is currently used : * dynamically by easyant to load system plugins (skeletons for example) * dynamically by easyant when you specify ea:build or ea:plugin tags in module.ivy files * invoked in plugin ant file itselfs * invoked in module.ant if users has complex needs Additionnal there is two alliases for this task to import plugins and buildtype. ea:plugin module=compile-java revision=0.9/ ea:plugin module=build-std-java revision=0.9/ If organisation attribute is not specified on aliases default one will be used. It does the job but it doesn't handle conflict between plugins. Some plugins relies on abstract ones. Exemple: package-jar depends on abstract-package, abstract-package depends on abstract-compile, but compile-java plugin also depends on abstract-compile. Which versi of abstract-compile will be taken in case both plugins load different version ? The answer is the first one ! This becomes more problematic on buildtypes, as buildtypes loads a set of plugins (including themself others abstract-plugins). Ok so now you should have a quick picture of the problem. What could be done ? We can rely on ivy to describe dependency on plugins. But then how could we know in which order plugins should be loaded ? I suggest to introduce a deferred import mechanism. We should split responsibility in two distinct steps. 1 - resolve (based on ivy) the whole graph of plugins and store the resolve report somewhere as a reusable reference in ant project 2 - invoke a new import task should import already resolved plugins (the task could rely on the report stored as a reference in ant project) Exemple : compile java will have an ivy dependency on abstract-compile dependency org=org.apache.easyant.plugins module=abstract-compile revision=1.0/ The compile java ant script will import the resolved plugin ea:import-deferred org=org.apache.easyant.plugins module=abstract-compile/ Note that i'm not fixed yet with the task name. Any suggestion (even for alliases are welcome). To maintain backward compatibility i'm in favor of creating new aliases import-plugin and import-buildtype instead of reusing existing ones. People would be able to continue using existing task with known the limitation (no conflict management on plugins). This can help if someone wants to load plugins in module.ant after setting a few properties. I also recommend adding a warning on existing task to recommend people using the new import mechanism. What do you think ?ea [1] https://issues.apache.org/jira/browse/EASYANT-13 -- Jean Louis Boudart Independent consultant Apache EasyAnt commiter http://ant.apache.org/easyant/
