> On 25 Apr 2020, at 16:31, Romain Manni-Bucau <[email protected]> wrote:
>
> Le sam. 25 avr. 2020 à 17:12, Robert Oxspring <[email protected]> a
> écrit :
>
>>
>>> On 25 Apr 2020, at 15:37, Romain Manni-Bucau <[email protected]>
>> wrote:
>>>
>>> Hi Robert, two thoughts maybe
>>>
>>> 1. all that work (thinking also of assembly one) should probably end up
>> by
>>> enriching
>>>
>> https://github.com/apache/maven-shared-incremental/blob/master/src/main/java/org/apache/maven/shared/incremental/IncrementalBuildHelper.java
>>> model
>>> and impl.
>>
>> Great! It’s been a couple of years since I’ve worked on a maven project so
>> pointers towards infrastructure pieces I’ve missed or forgotten is v
>> helpful! I’ll take a look at the incremental build helper.
>>
>>> I tend to think it does not need any hash but just a lastUpdated
>>> track (the most recent file invalidating previous cache, it is enough and
>>> faster than any hash computation)
>>
>> I’m not totally sure of this. Filtering, for example, is particularly
>> dependant on properties and configuration which aren’t directly linked with
>> a source file change. Allowing the feature to consider wider inputs should
>> make the feature more reliable, and might avoid the need to make it
>> optional. Definitely up for running the numbers and seeing the impact
>> before deciding though!
>>
>
> Hmm, more you'll put there, more you'll defeat your goal (didn't check on
> maven but did it for some "big" downloads where the size can be compared on
> maven project and hashing was a disaster if done globally).
> Globally if you depend on properties then you have to use more advanced
> hashing but also just rebuild generally so timestamp still work, you just
> force the rebuild in such a case.
> There is also the issue of plugin adding properties and downstream plugins
> using them, here you would need a state manager you reinject in
> maven...which will highly likely be wrong at the rebuild time.
> So maybe let's start simple?
This does take us somewhat full circle: The diff I linked to at the start of
this thread was a far simpler place to start. No state storage. Files only
modified if the content changes. Tbh, with this change in place I suspect I’d
be happy with lastModified driven incremental builds.
>>> 2. by default nothing should be skipped - I guess a
>> -Dxxx.incremental=true
>>> should be added, maybe even to maven core (see next point), all this
>> theory
>>> is based on fully defining inputs/outputs fully. This is what gradle
>>> implemented and I almost never saw a single complex gradle build
>> correctly
>>> configured to not bypass modules it shouldn't bypass (or worse, skip
>> tests
>>> it shouldn't) so I think we should be conservative here + several plugins
>>> refetch data when executed so even if the result of 1 is "no change" you
>>> can need to reexecute the plugin (including the filtering one if the
>>> filtered data depends on the time or so - I don't want to discuss it is a
>>> good or bad practise but it is used a lot in CI/CD pipelines).
>>
>> Interesting. To me the the default should be that nothing is repeated. Our
>> experiences of Gradle are entirely opposed too - I’ve never seen it’s
>> incremental build support doing the wrong thing. I’m well aware of the
>> necessity to support timestamps when filtering though.
>>
>
> At beginning i was wondering how Gradle was so fast...then i realized it
> was not doing its job. I didn't review enough projects probably (some
> dozen) but not a single one was well configured to ensure the build was
> deterministic until you kill the daemon and clean the project before the
> rebuild. It is very nasty and I hope maven does not get that kind of
> behavior *by default* (to explicit that '*': it is fine to me to get it in
> dev since on some projects it can help in dev iterations).
>
>
>>
>>> 3. Not being per plugin but global on maven sounds better than starting
>> to
>>> be specific in all plugins. I really see it as a dev feature you can
>>> activate while working and not enable on the CI for example. Added to
>>> maven-core it can enable to define in mojo the preconditions which would
>> be
>>> checked with 1 and potentially the conditions triggering inconditionally
>> a
>>> rebuild. Maybe a @MojoInput("${project.build.outputDirectory}",
>>> "${project.artifacts}", "method:getState()") or so.
>>
>> Yeah, that’s all looking much closer to what I’ve used in Gradle more
>> recently. It would make it a bigger piece of work, but I’m happy to explore
>> that if it’s the preferred option!
>>
>
> BTW, we should have started by that, but do you have figures about your
> project?
> Something like "each mojo taks X ms", i modify a class, estimated rebuild
> time is "Y ms" (maybe call the mojos manually, like "mvn compiler:compile
> jar:jar" etc), actual rebuild time with "mvn install" is "Z ms" (+
> drilldown per mojo).
Does seem a sensible to start. Is there a reasonable way to extract such timing
information? Was hoping that grepping A debut log would be sufficient but no
timings are obvious beyond the overall summary. Figuring out the relevant mojos
and manually timing for 30 modules Is pretty boring.
>
> Don't know if Hervé read this already but can't it be close to the
> reproducible track which has a kind of state - to make more generic
> probably but it sounds like a "standard"/existing option to investigate
> maybe?
Didn’t follow any of that but seems it was mostly for Hervé.
>
>
>>
>> Thanks for the input!
>>
>> Rob
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
