On Jan 10, 2015, at 3:16 AM, Mark Struberg <strub...@yahoo.de> wrote:
> Hi Martin! > > The maven-compiler plugin already does this. But once a single change is > detected then we need to recompile the _whole_ module because of the reasons > explained by Igor. > Pro of JDT: you can e.g. also see if there were only 'internal' changes and > thus the other dependencies don't need to recompile. > > I'm 99.8, but not 100% convinced by that. Probably the other dependencies > don't need any recompilation but maybe the whole internal behaviour changed > and you still like to run your tests? Maybe we can simply ignore this and > people should simply force a clean beforehand. > In the project where we developed the incremental lifecycle forcing a clean is not practical. It introduces a huge penalty when you're dealing with tens of source generators and annotations processors with hundreds of modules. Yes, it will take time to instrument all the plugins we care about but have done so thus far and it is possible to reason about the incremental state of the whole project as a consequence. Right now we are focused on individual mojo executions but eventually we want global incremental state management which is the path to full impact analysis. Reaching that state is when you can reason about what classes and resources changing affect what tests and you start running on the tests that are necessary. As Paul mentioned in the hangout he's got 20 hours of tests. In our projects we have 90 minutes functional tests on extremely powerful machines. But this is where the biggest gains will be made but a lot of work needs to be done to get to that point. > > Basically al the incremental stuff boils down to the fact that the outcome is > only determined by it's input. The problem is only to gather the right > 'input'. E.g. your packaging relies not only on the compiled classes but also > on resources and probably a manual include/exclude setting. Only the very > plugin used to do the packaging will know. Thus EACH plugin in the whole > chain needs to get adopted to incremental-awareness. Which is tons of work. > It is not only determined by inputs but also configuration, and not only changed inputs but deleted inputs in order to remove stale outputs: this is incredibly important. The other critical difference in our implementation is awareness of dependency relationships. To our knowledge we have the first, complete and correct incremental mechanism for Maven. Simon says Buck has something similar and I know of a couple commercial implementations. Our end goal is full impact analysis and something more akin to a global state machine. Yes, all the mojos you use must be instrumented correctly and consistently and this is a huge amount of work. For our customer we started with the default lifecycle, targeted plugins that were required to be instrumented and we're just stepwise improving the support. We had a requirement for pure speed in order to meet our wall time requirements but I believe the vast majority of people doing enterprise builds care about 20-30 of the plugins that exist and we'll eventually get coverage for those. > > Another story is if the 'input' e.g. includes data from a database or you > just like to re-run your unit tests for the sake of debugging. We need a > switch to force some plugins to run even if the plugin itself cannot see a > reason by simply looking at it's known 'input'. > There's nothing intrinsically different in this case. It's a matter of creating the correct graph and even if you had resources in databases or your schema changes you need to know how to process a delta from said resource and what is affected by those changes. There's nothing terribly magical here, the hard part for us right now is what the API should look like (we're having a lot of difficulty with that) and the sheer amount of work to instrument the plugins. This was definitely an oversight when we started writing the system. We didn't much think about incremental behaviour at all but it really is necessary. For us the benefit is that once a plugin is instrumented with our incremental API you get incremental behaviour on the CLI and inside of Eclipse and no special integration needs to be written. Ultimately from one API we want proper incremental behavior (and workspace resolution semantics) on the CLI, the IDE and CI systems. This is a requirement on the path to correct and efficient continuous delivery. It's all connected. > > LieGrue, > strub > > > > > >> On Saturday, 10 January 2015, 2:04, Martin Gainty <mgai...@hotmail.com> >> wrote: >>> G ood Evening Igor- >> >> Instead of implementing eclipe jdt for incremental java compilation is there: >> >> 1)any ability to refactor/reconfigure javac to read Last Modified Date of >> source >> file to enable incremental compilation? >> 2)any concerns over generated class files..are they the same? >> any deltas ..if so ...what are they? >> 3)can we implement a static settings.xml or other static configuration file >> that >> manfred suggested for eclipse jdt? >> 4)any ability to switch compiler from javac to jdt as an attribute in >> maven-compiler-plugin? >> >> Thanks for a great presentation on Maven Dev hangout >> Martin >> ______________________________________________ >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > Thanks, Jason ---------------------------------------------------------- Jason van Zyl Founder, Apache Maven http://twitter.com/jvanzyl http://twitter.com/takari_io --------------------------------------------------------- believe nothing, no matter where you read it, or who has said it, not even if i have said it, unless it agrees with your own reason and your own common sense. -- Buddha
