Well I just had a look and it seems as if the plugin does support retrying:
https://maven.apache.org/wagon/wagon-providers/wagon-http/ So from this: -Dmaven.wagon.http.retryHandler.requestSentEnabled=standard -Dmaven.wagon.http.retryHandler.count=6 Should allow enabling the retry mechanism the same way the maven-deploy-plugin is doing it. Chris Am 28.04.20, 19:28 schrieb "Alex Harui" <[email protected]>: I was unclear as to how to take advantage of that flag with Wagon. I thought deploy used wagon and not the other way around and didn't see a similar flag for Wagon. On 4/28/20, 10:24 AM, "Yishay Weiss" <[email protected]> wrote: I didn’t get that it was the number of tries per artifact. That could indeed be useful. From: Christofer Dutz<mailto:[email protected]> Sent: Tuesday, April 28, 2020 8:04 PM To: [email protected]<mailto:[email protected]> Subject: Re: [royale-asjs] branch release/0.9.7 updated: add retry task if wagon fails Hi Yishay, what we do in the PLC4X project (but only for SNAPSHOTS), is that on the CI server we tell it to deploy to a local directory. sh 'mvn -U -Pdevelopment,with-sandbox,with-logstash,with-c -DaltDeploymentRepository=snapshot-repo::default::file:./local-snapshots-dir clean deploy' We then use the wagon plugin to deploy. sh 'mvn -f jenkins.pom -X -P deploy-snapshots wagon:upload' The reason we're doing it that way is that we need our builds to run on a dedicated PLC4X VM which is managed by Infra because of some low-level-networking stuff we do in our tests. In order to deploy them to Nexus, we therefore have it deploy to a local directory and then have a node with credentials deploy that for us. Well I don't know if in your countries there are internet filters ... you probably know better ... from the technical side it sounds absolutely standard. And regarding running the deployment multiple times ... there is one huge difference though between running the entire deployment process multiple times or telling it to re-try an artifact 6 times. I would assume that if the chance to not deploy is at 1% (and I would like to claim it being a lot lower from my experience) and you have let's say 500 artifacts ... the chances are pretty slim to have the build pass as a whole. However if you don't fail the deployment if one artifact fails, but re-try every one up to 6 times, then the chances it passes are pretty high. In the end, if things would go south exactly 5 times for every artifact it would sort of be like if you tried running the bunch 6000 times. Chris Am 28.04.20, 18:21 schrieb "Yishay Weiss" <[email protected]>: Hi Chris, I just did an upload test and it came out 4.7mbps. Download came out 538kbps. I’m uploading from Israel, my ISP is Partner Communications, I’m using a WIFI network. My latest failure was half an hour ago (18:47 GMT +3). Thanks for pointing out the config option on the deployment plugin, though I’ve already tried more than 6 times. I’m wondering if it’s possible to package before uploading and resume on failure. There are lots of messages of transfers of ~40 bytes. I don’t know if they’re opening a separate connection for each of these transfers but it seems inefficient and it’s evidently fragile. Thanks, Yishay From: Christofer Dutz<mailto:[email protected]> Sent: Tuesday, April 28, 2020 4:59 PM To: [email protected]<mailto:[email protected]> Subject: Re: [royale-asjs] branch release/0.9.7 updated: add retry task if wagon fails Side-question ... So if you were using the maven wagon plugin (no matter if the apache one or the other) and were having issues, I would like to ask at infra if they know what they are and what might be causing them. Cause I have staged many releases on that server and never had issues (ok ... except that it took AAAAAAGES staging a PLC4X release in a plane). Perhaps there are some infrastructural issues like web-filters etc. that could be interfering. So could you please tell me (if you want in off-list) from which country you were staging things (perhaps the type (DSL, Cable, ...) ... how many Mbit bandwith and name of your internet-provider). Also the time you had problems could be important (perhaps we can see something in the logs). And did you see the config option I added to the deploy plugin? <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-deploy-plugin</artifactId> <version>2.8.2</version> <!-- If deploying fails due to repo or network problems, retry the given number of times (1-10) --> <configuration> <retryFailedDeploymentCount>6</retryFailedDeploymentCount> </configuration> </plugin> Perhaps that could help? Chris Am 28.04.20, 10:57 schrieb "Yishay Weiss" <[email protected]>: I think it should be possible to do this in powershell, with: $password = Read-Host "Enter Password" -AsSecureString And pass $pass as the param. Are you suggesting to upload the .asc files separately from the local computer? From: Alex Harui<mailto:[email protected]> Sent: Tuesday, April 28, 2020 9:29 AM To: [email protected]<mailto:[email protected]> Subject: Re: [royale-asjs] branch release/0.9.7 updated: add retry task if wagon fails I think I have found a way to get the CI Server to upload the artifacts without storing credentials on the CI server. The settings.xml file has the following entries for the servers: <username>${rm.username}</username> <password>${rm.password}</password> Now we need a secure way to have the RM enter their Apache password and run a command like: mvn wagon:upload -Drm.username=... -Drm.password=... ... I tested it by typing my password on the command line but it is visible. That is a bit risky in case someone else logs in and kicks you off RDP. Does someone know PowerShell and can come up with a script that asks for the password and hides it and passes it on to Maven without making it visible? There are some articles on doing it in .BAT but they seem like a hack. I suppose we could write an AIR app that does it or some other Windows EXE? Or install a Bash shell. Thoughts? -Alex On 4/27/20, 10:09 PM, "Alex Harui" <[email protected]> wrote: We are currently using org.codehaus.mojo's wagon-maven-plugin 2.0.0. There is an org.apache.maven.wagon.Wagon that is at 3.4.0. I think we can switch to that by replacing the wagon entry in the royale-asjs pom.xml <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>wagon-maven-plugin</artifactId> <version>2.0.0</version> <configuration> <includes>**</includes> <serverId>apache.releases.https</serverId> <url>https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fservice%2Flocal%2Fstaging%2Fdeploy%2Fmaven2&data=02%7C01%7Caharui%40adobe.com%7C6cd664bf1d564df9077d08d7eb990f27%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637236914935820254&sdata=q5HTwadLsXvf5v4nIF3lri5A6p0kejL9Dd%2FhBI38mSA%3D&reserved=0</url> </configuration> </plugin> Would become something like: <plugin> <groupId>org.apache.maven.wagon </groupId> <artifactId>wagon</artifactId> <version>3.4.0</version> <configuration> <includes>**</includes> <serverId>apache.releases.https</serverId> <url>https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fservice%2Flocal%2Fstaging%2Fdeploy%2Fmaven2&data=02%7C01%7Caharui%40adobe.com%7C6cd664bf1d564df9077d08d7eb990f27%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637236914935820254&sdata=q5HTwadLsXvf5v4nIF3lri5A6p0kejL9Dd%2FhBI38mSA%3D&reserved=0</url> </configuration> </plugin> HTH, -Alex On 4/27/20, 1:46 PM, "Alex Harui" <[email protected]> wrote: Don't see Wagon 3.4.0 on Central. I do see Wagon Providers 3.4.0. How would we get the Wagon we use to use Wagon Providers? When the upload fails, Yishay is not given a away to start from the point of failure ("maven -rf ..."). I have suggested using Wagon to upload individual folders -Alex On 4/27/20, 3:07 AM, "Christofer Dutz" <[email protected]> wrote: Hi all, a week ago a new version of Wagon was released ... perhaps worth updating to 3.4.0? Chris Am 27.04.20, 11:58 schrieb "Yishay Weiss" <[email protected]>: Awesome ________________________________ From: [email protected] <[email protected]> Sent: Monday, April 27, 2020 8:56:00 AM To: [email protected] <[email protected]> Subject: [royale-asjs] branch release/0.9.7 updated: add retry task if wagon fails This is an automated email from the ASF dual-hosted git repository. aharui pushed a commit to branch release/0.9.7 in repository https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitbox.apache.org%2Frepos%2Fasf%2Froyale-asjs.git&data=02%7C01%7Caharui%40adobe.com%7C6cd664bf1d564df9077d08d7eb990f27%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637236914935820254&sdata=WQaS%2FITTZRVgANXjC54Vw00wRqZoNUFlzp4IJ%2F6kUhw%3D&reserved=0 The following commit(s) were added to refs/heads/release/0.9.7 by this push: new adeb2e3 add retry task if wagon fails adeb2e3 is described below commit adeb2e3782b59342d9132adccaf357aeece32475 Author: Alex Harui <[email protected]> AuthorDate: Sun Apr 26 22:55:41 2020 -0700 add retry task if wagon fails --- releasesteps.xml | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/releasesteps.xml b/releasesteps.xml index 729a6fc..d0f3f0a 100644 --- a/releasesteps.xml +++ b/releasesteps.xml @@ -836,6 +836,23 @@ <arg value="_releasesteps-upload-release-to-staging" /> </exec> </target> + <target name="retryUploadSWCs" depends="get-artifact-folder" description="upload all SWCs except Jewel Themes"> + <fail message="release.version not specified" + unless="release.version"/> + <fail message="rf_arg not specified" + unless="rf_arg"/> + <exec executable="${mvn}" dir="${artifactfolder}/sources" failonerror="true" > + <arg value="wagon:upload" /> + <arg value="-X" /> + <arg value="-rf" /> + <arg value="${rf_arg}" /> + <arg value="-Dwagon.fromDir=${artifactfolder}/artifacts/archive/target/local-release-dir" /> + <!--arg value="-Dwagon.includes=**" /> + <arg value="-Dwagon.excludes=**/Jewel-Dark*/**,**/Jewel-Light*/**" /--> + <arg value="-P" /> + <arg value="_releasesteps-upload-release-to-staging" /> + </exec> + </target> <target name="uploadJewelDark" depends="get-artifact-folder" description="upload Jewel Dark Themes"> <fail message="release.version not specified" unless="release.version"/>
