Re: [I] Is mvnd "safe" for parallel builds where "mvn" alone is not? [maven-mvnd]
cstamas commented on issue #896: URL: https://github.com/apache/maven-mvnd/issues/896#issuecomment-1774124644 Split local repository is new feature, but you have to be aware that in Maven 3 land not all plugins "play nice" with it, see https://issues.apache.org/jira/browse/MNG-7706 In short, if using Maven 3.9.x and you do not see "plugin validation warnings" (see https://maven.apache.org/guides/plugins/validation/index.html), then you should be pretty much okay (but still no 100% guarantee). Best is to try locally and see first (so "lab testing" is what I'd recommend). Moreover, as split local repository does know to be a bit "mind boggling", I'd really even more recommend to play with it locally (on dev workstation), and when all in place, and if all OK, apply that to CI. mvnd := `mvn 3.9.x` (or 4, edition dep) + `resolver 1.9.x` + `smart builder` + `concurrent logging` + `resident daemon`. So in short, "yes, mvnd knows all what mvn 3.9.x plus much more". In mvnd the file locking is enabled by default (due resident daemon processes sharing same local repository). Split repo for that use case is next on my roadmap, but gonna happen only in Resolver 2.0 (so Maven 4 final), not in Maven 3.9.x lifecycle. Currently, "split" can only split based as documented: origin remote repo and cached vs install. Current goal of it was "branched development" (one local repo shared with several maven processes building same project but different branches of it). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Is mvnd "safe" for parallel builds where "mvn" alone is not? [maven-mvnd]
jimklimov commented on issue #896: URL: https://github.com/apache/maven-mvnd/issues/896#issuecomment-1774122880 And circling back to this repository's topic, with some new knowledge in mind - does current `mvnd` benefit from the new concurrency-safe resolver like maven 3.9.x does? A large part of my team members' question boils down to whether we can replace the original maven by `mvnd`+`mvn` frontend just at a finger-snap, by putting different tools into the PATH and not changing much more in the pipelines etc. -- and if this would bring some efficiency benefits? Originally this idea to use `mvnd` came up with brainstorming a one-off case of SBOM processing script that does a lot of analytics over `mvn help:effective-pom` (each call takes some 3 seconds to generate an XML, which adds up to evil run-times for hundreds of components in a deliverable bundle). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Is mvnd "safe" for parallel builds where "mvn" alone is not? [maven-mvnd]
jimklimov commented on issue #896: URL: https://github.com/apache/maven-mvnd/issues/896#issuecomment-1774121080 While at it, I am trying to wrap my head around the https://maven.apache.org/resolver/local-repository.html#split-local-repository feature. Is there some trick that would allow several CI builds to share downloaded third-party artifacts but let them store separately and use without conflict some artifacts from designated "our" namespaces? I saw suggestions about e.g. `-Daether.enhancedLocalRepository.localPrefix=$PROJECT/$BRANCH` - but it seems more related to where `mvn install` would land. Things like `mvn test` happen inside the build workspace and do not write the built code/test binaries into the local-repo on their own accord, right? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Is mvnd "safe" for parallel builds where "mvn" alone is not? [maven-mvnd]
jimklimov commented on issue #896: URL: https://github.com/apache/maven-mvnd/issues/896#issuecomment-1773940773 Thanks for the options, my first shot missed the "file-gav" part ;) Putting them into SCM as part of the components' source seems a bit like overkill... they should be buildable anywhere (and maybe with different environmental settings), right? For posterity and my own back-tracking, I'll exploring the `MAVEN_OPTS` envvar instead for now, so each Jenkins worker might set what is relevant there... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Is mvnd "safe" for parallel builds where "mvn" alone is not? [maven-mvnd]
cstamas commented on issue #896: URL: https://github.com/apache/maven-mvnd/issues/896#issuecomment-1773936232 create (and check in into SCM) a file in project like this: ``` .mvn/maven.config ``` with contents ``` -Daether.syncContext.named.factory=file-lock -Daether.syncContext.named.nameMapper=file-gav ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Is mvnd "safe" for parallel builds where "mvn" alone is not? [maven-mvnd]
jimklimov commented on issue #896: URL: https://github.com/apache/maven-mvnd/issues/896#issuecomment-1773928553 Oh that funny moment when the huge internet looks like a small village: https://www.mail-archive.com/users@maven.apache.org/msg144072.html > Configuration should be as easy as setting “aether.syncContext.named.factory” to “file-lock” So looking for some best way among several possibilities, to pass `aether.syncContext.named.factory=file-lock` from CI to maven and not confuse possible other (pre 3.9.x) tool versions along the way :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Is mvnd "safe" for parallel builds where "mvn" alone is not? [maven-mvnd]
cstamas commented on issue #896: URL: https://github.com/apache/maven-mvnd/issues/896#issuecomment-1773851213 Maven 3.9 introduced "locks" for local repository, trying to solve exactly that: shared access to local repository from multiple processes... So best would be to try it out (hopefully using some sound FS like ext4 or alike, no windows in picture). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Is mvnd "safe" for parallel builds where "mvn" alone is not? [maven-mvnd]
jimklimov commented on issue #896: URL: https://github.com/apache/maven-mvnd/issues/896#issuecomment-1773848683 Thanks for suggestions, checked that the workers used maven 3.8.6... So change to 3.9.x could just fix the situation while keeping all those build calls independent as they are now? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Is mvnd "safe" for parallel builds where "mvn" alone is not? [maven-mvnd]
cstamas commented on issue #896: URL: https://github.com/apache/maven-mvnd/issues/896#issuecomment-1773845614 First, please forget the "built in" parallel builder, use instead this https://github.com/takari/takari-smart-builder (same is used by mvnd). Second, did you try maven 3.9.x (preferably latest) version? You could use file locking for start (if local repository is on local FS) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
