Le sam. 25 avr. 2020 à 18:38, Robert Oxspring <roxspr...@imapmail.org> a écrit :
> > > On 25 Apr 2020, at 16:31, Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > > > > Le sam. 25 avr. 2020 à 17:12, Robert Oxspring <roxspr...@imapmail.org> > a > > écrit : > > > >> > >>> On 25 Apr 2020, at 15:37, Romain Manni-Bucau <rmannibu...@gmail.com> > >> wrote: > >>> > >>> Hi Robert, two thoughts maybe > >>> > >>> 1. all that work (thinking also of assembly one) should probably end up > >> by > >>> enriching > >>> > >> > https://github.com/apache/maven-shared-incremental/blob/master/src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java > >>> model > >>> and impl. > >> > >> Great! It’s been a couple of years since I’ve worked on a maven project > so > >> pointers towards infrastructure pieces I’ve missed or forgotten is v > >> helpful! I’ll take a look at the incremental build helper. > >> > >>> I tend to think it does not need any hash but just a lastUpdated > >>> track (the most recent file invalidating previous cache, it is enough > and > >>> faster than any hash computation) > >> > >> I’m not totally sure of this. Filtering, for example, is particularly > >> dependant on properties and configuration which aren’t directly linked > with > >> a source file change. Allowing the feature to consider wider inputs > should > >> make the feature more reliable, and might avoid the need to make it > >> optional. Definitely up for running the numbers and seeing the impact > >> before deciding though! > >> > > > > Hmm, more you'll put there, more you'll defeat your goal (didn't check on > > maven but did it for some "big" downloads where the size can be compared > on > > maven project and hashing was a disaster if done globally). > > Globally if you depend on properties then you have to use more advanced > > hashing but also just rebuild generally so timestamp still work, you just > > force the rebuild in such a case. > > There is also the issue of plugin adding properties and downstream > plugins > > using them, here you would need a state manager you reinject in > > maven...which will highly likely be wrong at the rebuild time. > > So maybe let's start simple? > > This does take us somewhat full circle: The diff I linked to at the start > of this thread was a far simpler place to start. No state storage. Files > only modified if the content changes. Tbh, with this change in place I > suspect I’d be happy with lastModified driven incremental builds. > > >>> 2. by default nothing should be skipped - I guess a > >> -Dxxx.incremental=true > >>> should be added, maybe even to maven core (see next point), all this > >> theory > >>> is based on fully defining inputs/outputs fully. This is what gradle > >>> implemented and I almost never saw a single complex gradle build > >> correctly > >>> configured to not bypass modules it shouldn't bypass (or worse, skip > >> tests > >>> it shouldn't) so I think we should be conservative here + several > plugins > >>> refetch data when executed so even if the result of 1 is "no change" > you > >>> can need to reexecute the plugin (including the filtering one if the > >>> filtered data depends on the time or so - I don't want to discuss it > is a > >>> good or bad practise but it is used a lot in CI/CD pipelines). > >> > >> Interesting. To me the the default should be that nothing is repeated. > Our > >> experiences of Gradle are entirely opposed too - I’ve never seen it’s > >> incremental build support doing the wrong thing. I’m well aware of the > >> necessity to support timestamps when filtering though. > >> > > > > At beginning i was wondering how Gradle was so fast...then i realized it > > was not doing its job. I didn't review enough projects probably (some > > dozen) but not a single one was well configured to ensure the build was > > deterministic until you kill the daemon and clean the project before the > > rebuild. It is very nasty and I hope maven does not get that kind of > > behavior *by default* (to explicit that '*': it is fine to me to get it > in > > dev since on some projects it can help in dev iterations). > > > > > >> > >>> 3. Not being per plugin but global on maven sounds better than starting > >> to > >>> be specific in all plugins. I really see it as a dev feature you can > >>> activate while working and not enable on the CI for example. Added to > >>> maven-core it can enable to define in mojo the preconditions which > would > >> be > >>> checked with 1 and potentially the conditions triggering > inconditionally > >> a > >>> rebuild. Maybe a @MojoInput("${project.build.outputDirectory}", > >>> "${project.artifacts}", "method:getState()") or so. > >> > >> Yeah, that’s all looking much closer to what I’ve used in Gradle more > >> recently. It would make it a bigger piece of work, but I’m happy to > explore > >> that if it’s the preferred option! > >> > > > > BTW, we should have started by that, but do you have figures about your > > project? > > Something like "each mojo taks X ms", i modify a class, estimated rebuild > > time is "Y ms" (maybe call the mojos manually, like "mvn compiler:compile > > jar:jar" etc), actual rebuild time with "mvn install" is "Z ms" (+ > > drilldown per mojo). > > Does seem a sensible to start. Is there a reasonable way to extract such > timing information? Was hoping that grepping A debut log would be > sufficient but no timings are obvious beyond the overall summary. Figuring > out the relevant mojos and manually timing for 30 modules Is pretty boring. > Guess you can implement a log parser quickly and extract it from there enabling time logging or just grab a profiler maybe? > > > > > Don't know if Hervé read this already but can't it be close to the > > reproducible track which has a kind of state - to make more generic > > probably but it sounds like a "standard"/existing option to investigate > > maybe? > > Didn’t follow any of that but seems it was mostly for Hervé. > Yep, overall idea is reproducible builds require to be able to check a previous build state. For now it is mainly the output but I suspect the input is not that far so I just wonder if we should target to converge at some point or not. Really an open question ATM. > > > > > > >> > >> Thanks for the input! > >> > >> Rob > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >> For additional commands, e-mail: dev-h...@maven.apache.org > >> > >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >
