Re: Oak Jenkins build takes up too many resources

On 16.12.22, 20:01, "Konrad Windszus" wrote: > @Marcel: Are you still looking into optimizations? No, I’m not. Regards Marcel

Re: Oak Jenkins build takes up too many resources

Maybe adding some logic to only build modified modules helps like outlined in https://github.com/mrguamos/jenkins-pipeline-maven-multi-module/blob/master/Jenkinsfile . Hard though to figure out transitive

Re: Oak Jenkins build takes up too many resources

The situation is still pretty bad. Right now Oak jobs take up roughly 300 jobs of total 450 in the queue. We should ASAP switch to incremental builds (i.e. only trigger builds for modules which really have changed) and no longer trigger 100s of build jobs in parallel for a single branch!

Re: Oak Jenkins build takes up too many resources

Hi Marcel, Hi Konrad, Thanks for looking into this, some good improvements are on the way, and it seems like the problem severity was inflated by the way Jenkins reports build step duration. Thanks, Robert On Tue, 2022-11-15 at 10:56 +0100, Konrad Windszus wrote: > Probably it would also be

Re: Oak Jenkins build takes up too many resources

Hi, On 11.11.22, 15:11, "Robert Munteanu" wrote: > I think it would be worthile to check if the build can be optimised FYI, I also changed the Job configuration to not build PRs in draft state. Regards Marcel

Re: Oak Jenkins build takes up too many resources

Probably it would also be fair to not parallelise all modules (i.e. reserve only handful number of slots instead of almost all available). Currently we trigger 35 builds at the same time: https://github.com/apache/jackrabbit-oak/blob/6c04951be723c4574d3a7c1bcc4ec04e9f9e7dd0/Jenkinsfile#L23

Re: Oak Jenkins build takes up too many resources

On 14.11.22, 16:29, "Konrad Windszus" wrote: > Can you come up with a PR? See https://github.com/apache/jackrabbit-oak/pull/755 Though, I don’t think the build it triggers actually uses the updated Jenkinsfile. Regards Marcel

Re: Oak Jenkins build takes up too many resources

Seems this is pretty easy: https://stackoverflow.com/a/70375236 Can you come up with a PR? Thanks, Konrad > On 14. Nov 2022, at 16:16, Marcel Reutegger > wrote: > > Hi, > > On 11.11.22, 15:11, "Robert Munteanu" wrote: >> I think it would be worthile to

Re: Oak Jenkins build takes up too many resources

Hi, On 11.11.22, 15:11, "Robert Munteanu" wrote: > I think it would be worthile to check if the build can be optimised, > otherwise Oak builds are blocking many execution slots of the Jenkins > ASF instance. I did notice one thing. Our PRs may schedule a build each time commits are pushed to

Re: Oak Jenkins build takes up too many resources

Hi Robert, On 11.11.22, 15:11, "Robert Munteanu" wrote: > I was waiting for a small PR of mine in Sling to be built by Jenkins and > I saw that it was not picked for some time. Looking at the executing > jobs I noticed that there were many, different, Oak stages being ran. > > A sample PR check

Oak Jenkins build takes up too many resources

Hi, I was waiting for a small PR of mine in Sling to be built by Jenkins and I saw that it was not picked for some time. Looking at the executing jobs I noticed that there were many, different, Oak stages being ran. A sample PR check [1] shows that there are many stages (one per module?) but