Romain - as far as I understand, Maven *does* have a retry strategy, but it is a poor retry strategy and there is no way to tweak it. In particular, if it uses https://maven.apache.org/guides/mini/guide-http-settings.html , that means it uses Apache Http Client 4.1.2 whose default retry settings are extremely conservative https://github.com/apache/httpcomponents-client/blob/4.1.x/httpclient/src/main/java/org/apache/http/impl/client/DefaultHttpRequestRetryHandler.java#L93
The errors we're hitting with Maven are some of those: "connection refused" and alike. On Mon, Dec 4, 2017 at 12:15 PM Romain Manni-Bucau <[email protected]> wrote: > 2017-12-04 18:58 GMT+01:00 Kenneth Knowles <[email protected]>: > > On Sat, Dec 2, 2017 at 3:06 AM, Romain Manni-Bucau < > [email protected]> > > wrote: > >> > >> Need to check but if plugin dependencies were not tuned it should be the > >> default with a retry of 3 IIRC. > >> > >> But arent you sure it was a repo/server issue any client cant solve? > > > > > > Not sure what you mean here. What we need is for mvn and gradle to > succeed > > even though there are frequent transient failures of the repo. Anything > that > > makes this happen is success. > > What I meant is you will likely never get it with whatever build > system until you go 100% local which wouldn't make the build > reproducible. > > Concretely you can do > > 1. > > for (int i = 0; i < 3; i++) { > if (download()) break; > } > > > 2. > > for (int i = 0; i < 3; i++) { > if (download()) break; > exponentialBackoff(); > } > > > Or other wait strategy with a timeout. > but you can't have: > > while (!download()) { > waitUntilItWorks(); > } > > > So concretely if your down time > reasonnable time you will see the > failure anyway. > > Maven default has a retry (you see it sometimes in the logs) so should > tolerante a restart or small downtime. Ivy (gradle) uses httpclient as > well so same kind of config is available but with the same limitation > by design. > > Having a local cache is the best way to avoid these issues but it > requires at least one green build for dependencies. > > What I often saw was to do a pre-build step just resolving > dependencies+plugins and use a .m2/repository caching. > > > > > Kenn > > > >> > >> > >> Le 1 déc. 2017 23:27, "Kenneth Knowles" <[email protected]> a écrit : > >>> > >>> How do you instruct maven or gradle to use that all the time? > >>> > >>> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau > >>> <[email protected]> wrote: > >>>> > >>>> If you use wagon http - and not lightweigh - it should have retries by > >>>> default. > >>>> > >>>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <[email protected]> a > >>>> écrit : > >>>>> > >>>>> Has this ever been brought up in Maven dev community? Perhaps they > have > >>>>> some suggestions. It sounds like a reasonable feature request. > >>>>> > >>>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <[email protected]> > wrote: > >>>>>> > >>>>>> I've repeatedly searched around for a way to just add proper retry > to > >>>>>> maven or gradle, and haven't found anything :-( > >>>>>> > >>>>>> I had thought that we altered our builds in such a way that the .m2 > >>>>>> directory was permitted to survive across builds. True that it isn't > >>>>>> hermetic, precisely, but it is pretty safe to treat as a cache of > immutable > >>>>>> data, which is no more dangerous than having a caching incremental > build > >>>>>> system. > >>>>>> > >>>>>> Kenn > >>>>>> > >>>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov > >>>>>> <[email protected]> wrote: > >>>>>>> > >>>>>>> Our builds often hit transient Maven network issues, e.g. this one > >>>>>>> > >>>>>>> > https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/5331/consoleFull > >>>>>>> > >>>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project > >>>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve > dependencies for > >>>>>>> project > >>>>>>> > org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests:jar:2.3.0-SNAPSHOT: > >>>>>>> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0 > from/to > >>>>>>> central (https://repo.maven.apache.org/maven2): GET request of: > >>>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central > failed: > >>>>>>> Connection reset -> [Help 1] > >>>>>>> > >>>>>>> It'd be good to increase reliability of our builds. > >>>>>>> repo.maven.apache.org seems quite unreliable. > >>>>>>> > >>>>>>> I tried finding a way to configure Maven to retry such network > errors > >>>>>>> and it appears to be impossible [will be happy if someone proves > me wrong]. > >>>>>>> > >>>>>>> Would this issue be resolved if we used multiple mirrors? > >>>>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html > Any other > >>>>>>> suggestions? > >>>>>> > >>>>>> > >>>>> > >>> > > >
