[ https://issues.apache.org/jira/browse/MBUILDCACHE-59?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bas van Erp updated MBUILDCACHE-59: ----------------------------------- Description: I would argue that the Maven Build Cache Extension (MBCE) can provide the greatest value for users that utilize large multi-module (mono-)repositories, since those projects tend to have large amounts of "stable" code which isn't modified frequently. Speeding up the build of these multi-module projects using the MBCE would be totally awesome! But for my particular project (200K LOC, 400 modules, mono-repo) our initial trials with the MBCE have proven somewhat troublesome. It has to do with our use of [https://maven.apache.org/maven-ci-friendly.html] h2. Example setup {{repo/.mvn/maven.config}} {code:java} -Drevision=0.0.0-local-SNAPSHOT {code} {{repo/pom.xml}} {code:java} <groupId>com.corp</groupId> <artifactId>parent</artifactId> <version>${revision}</version> <packaging>pom</packaging> {code} {{repo/moduleX/pom.xml}} {code:java} <parent> <groupId>com.corp</groupId> <artifactId>parent</artifactId> <version>${revision}</version> <relativePath>../pom.xml</relativePath> </parent> <artifactId>moduleX</artifactId> <packaging>jar</packaging>{code} {{repo/moduleY/pom.xml}} {code:java} <parent> <groupId>com.corp</groupId> <artifactId>parent</artifactId> <version>${revision}</version> <relativePath>../pom.xml</relativePath> </parent> <artifactId>moduleY</artifactId> <packaging>jar</packaging> <dependencies> <dependency> <groupId>com.corp</groupId> <artifactId>moduleX</artifactId> <version>${revision}</version> </dependency> </dependencies>{code} {{repo/Jenkinsfile}} {code:java} mvn clean deploy -Drevision=$(git describe){code} h2. Effect This setup effectively means that while all our 150 developers might theoretically get decent cache hit % since they are all always using version {{0.0.0-local-SNAPSHOT}} for every single POM for every single build, our CI pipelines are out of luck. Since the effective-POMs will change radically with each new Git commit, the hashes will virtually never collide. h2. Solution? I'm very new to the MBCE so I might be way off the mark, but I think accommodating this setup will not be trivial to implement. It would probably require parsing the effective-POM for every module picked-up by the reactor and then removing all <version> clauses of the module, the parent, and each dependency and plugin which are part of the reactor build. Doesn't sound easy to me. But perhaps I'm missing something. p.s. Congrats on the v1.0.0 release :). I'm enjoying experimenting with it. But the documentation could use some love. It's pretty hard to read. Some paragraphs are confusing, terse or lacking in detail, are clearly written by non-English-native writers (like myself), or even stop mid-sentence. was: I would argue that the Maven Build Cache Extension (MBCE) can provide the greatest value for users that utilize large multi-module (mono-)repositories, since those projects tend to have large amounts of "stable" code which isn't modified frequently. Speeding up the build of these multi-module projects using the MBCE would be totally awesome! But for my particular project (200K LOC, 400 modules, mono-repo) our initial trials with the MBCE have proven somewhat troublesome. It has to do with our use of [https://maven.apache.org/maven-ci-friendly.html] h2. Example setup {{repo/.mvn/maven.config}} {code:java} -Drevision=0.0.0-local-SNAPSHOT {code} {{repo/pom.xml}} {code:java} <groupId>com.corp</groupId> <artifactId>parent</artifactId> <version>${revision}</version> <packaging>pom</packaging> {code} {{repo/moduleX/pom.xml}} {code:java} <parent> <groupId>com.corp</groupId> <artifactId>parent</artifactId> <version>${revision}</version> <relativePath>../pom.xml</relativePath> </parent> <artifactId>moduleX</artifactId> <packaging>jar</packaging>{code} {{repo/moduleY/pom.xml}} {code:java} <parent> <groupId>com.corp</groupId> <artifactId>parent</artifactId> <version>${revision}</version> <relativePath>../pom.xml</relativePath> </parent> <artifactId>moduleY</artifactId> <packaging>jar</packaging> <dependencies> <dependency> <groupId>com.corp</groupId> <artifactId>moduleX</artifactId> <version>${revision}</version> </dependency> </dependencies>{code} {{repo/Jenkinsfile}} {code:java} mvn clean deploy -Drevision=$(git describe){code} h2. Effect This setup effectively means that while all our 150 developers might theoretically get decent cache hit % since they are all always using version {{0.0.0-local-SNAPSHOT}} for every single POM for every single build, our CI pipelines are out of luck. Since the effective-POMs will change radically with each new Git commit, the hashes will virtually never collide. h2. Solution? I'm very new to the MBCE so I might be way off the mark, but I think accommodating this setup will not be trivial to implement. It would probably require parsing the effective-POM for every module picked-up by the reactor and then removing all <version> clauses of the module, the parent, and each dependency and plugin which are part of the reactor build. Doesn't sound easy to me. But perhaps I'm missing something. p.s. Congrats on the v1.0.0 release :). I'm enjoying experimenting with it. But the documentation could use some love. It's pretty hard to read. Some paragraphs are confusing, lacking in detail, are clearly written by non-English-native writers (like myself), or even stop mid-sentence. > Accommodate Maven CI-friendly versions > -------------------------------------- > > Key: MBUILDCACHE-59 > URL: https://issues.apache.org/jira/browse/MBUILDCACHE-59 > Project: Maven Build Cache Extension > Issue Type: Improvement > Affects Versions: 1.0.0 > Reporter: Bas van Erp > Priority: Major > > I would argue that the Maven Build Cache Extension (MBCE) can provide the > greatest value for users that utilize large multi-module (mono-)repositories, > since those projects tend to have large amounts of "stable" code which isn't > modified frequently. > Speeding up the build of these multi-module projects using the MBCE would be > totally awesome! > But for my particular project (200K LOC, 400 modules, mono-repo) our initial > trials with the MBCE have proven somewhat troublesome. > It has to do with our use of [https://maven.apache.org/maven-ci-friendly.html] > h2. Example setup > {{repo/.mvn/maven.config}} > {code:java} > -Drevision=0.0.0-local-SNAPSHOT {code} > {{repo/pom.xml}} > {code:java} > <groupId>com.corp</groupId> > <artifactId>parent</artifactId> > <version>${revision}</version> > <packaging>pom</packaging> {code} > {{repo/moduleX/pom.xml}} > {code:java} > <parent> > <groupId>com.corp</groupId> > <artifactId>parent</artifactId> > <version>${revision}</version> > <relativePath>../pom.xml</relativePath> > </parent> > <artifactId>moduleX</artifactId> > <packaging>jar</packaging>{code} > {{repo/moduleY/pom.xml}} > {code:java} > <parent> > <groupId>com.corp</groupId> > <artifactId>parent</artifactId> > <version>${revision}</version> > <relativePath>../pom.xml</relativePath> > </parent> > <artifactId>moduleY</artifactId> > <packaging>jar</packaging> > <dependencies> > <dependency> > <groupId>com.corp</groupId> > <artifactId>moduleX</artifactId> > <version>${revision}</version> > </dependency> > </dependencies>{code} > {{repo/Jenkinsfile}} > {code:java} > mvn clean deploy -Drevision=$(git describe){code} > h2. Effect > This setup effectively means that while all our 150 developers might > theoretically get decent cache hit % since they are all always using version > {{0.0.0-local-SNAPSHOT}} for every single POM for every single build, our CI > pipelines are out of luck. > Since the effective-POMs will change radically with each new Git commit, the > hashes will virtually never collide. > h2. Solution? > I'm very new to the MBCE so I might be way off the mark, but I think > accommodating this setup will not be trivial to implement. It would probably > require parsing the effective-POM for every module picked-up by the reactor > and then removing all <version> clauses of the module, the parent, and each > dependency and plugin which are part of the reactor build. > Doesn't sound easy to me. But perhaps I'm missing something. > > p.s. Congrats on the v1.0.0 release :). I'm enjoying experimenting with it. > But the documentation could use some love. It's pretty hard to read. Some > paragraphs are confusing, terse or lacking in detail, are clearly written by > non-English-native writers (like myself), or even stop mid-sentence. -- This message was sent by Atlassian Jira (v8.20.10#820010)