[jira] [Comment Edited] (WAGON-575) Retry for connection issues
[
https://issues.apache.org/jira/browse/WAGON-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277726#comment-17277726
]
Igor Dvorzhak edited comment on WAGON-575 at 2/3/21, 6:31 AM:
--
[~michael-o] Yes, I think that 8u and 11u both are affected - this bug was
introduced in [JDK-8145252|https://bugs.openjdk.java.net/browse/JDK-8145252]
that added TLS v1.3 support in JDK 8 (after backport) and 11+
was (Author: medb):
[~michael-o] Yes, I think that 8u and 11u both are affected - this bug was
introduced in [JDK-8145252|https://bugs.openjdk.java.net/browse/JDK-8145252]
that added TLS v1.3 support in JDK 8 and 11+
> Retry for connection issues
> ---
>
> Key: WAGON-575
> URL: https://issues.apache.org/jira/browse/WAGON-575
> Project: Maven Wagon
> Issue Type: New Feature
> Components: wagon-http
>Affects Versions: 3.3.4
> Environment: windows and linux
>Reporter: Michael Medin
>Priority: Minor
>
> There are a RetryHandler and now also a ServiceRetryHandler but both seem to
> focus only the handshake and no the data stream.
> In our case we download large artifacts (1+Gb) over a sometimes shaky
> connection which causes frequent "Connection reset" issues.
> To mitigate this we started to implement retry logic based on this
> [https://maven.apache.org/wagon/wagon-providers/wagon-http/] document. But
> seems our retry handler is never invoked when the connection is reset.
> So after some digging into the source code it seems the retryhandler are only
> used when connecting to the server and once the connection has been
> established and the HTTP headers have been read there is no retry handling
> for the reminder stream copy.
> Looking at the code it seems non trivial to add retry for this at is split up
> so I wanted to know if I am missing something before looking at implementing
> a PR for this.
>
> A simple way to simulate this behavior is to start a maven build with some
> large dependencies and during the download phase just kill the internet
> connection.
> If you have a RetryHandler enabled it will never be called instead you will
> get a stack trace along the following:
> {{[ERROR] Failed to execute goal on project test: Could not resolve
> dependencies for project test:test:jar:1.0.0: Could not transfer artifact
> REDACTED from/to REDACTED from REDACTED failed: Connection reset -> [Help 1]}}
> {{org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project test: Could not resolve dependencies for project
> test:test:jar:1.0.0: Could not transfer artifact REDACTED from/to REDACTED
> (REDACTED): GET request of: REDACTED from REDACTED failed}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
> (LifecycleDependencyResolver.java:269)}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
> (LifecycleDependencyResolver.java:147)}}
> {{ at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
> (MojoExecutor.java:248)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:202)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)}}
> {{ at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)}}
> {{ at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)}}
> {{ at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)}}
> {{ at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)}}
> {{ at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)}}
> {{ at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)}}
> {{ at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)}}
> {{ at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke (Method.java:566)}}
> {{ at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:282)}}
> {{ at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:225)}}
> {{
[jira] [Comment Edited] (WAGON-575) Retry for connection issues
[
https://issues.apache.org/jira/browse/WAGON-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245396#comment-17245396
]
Michael Osipov edited comment on WAGON-575 at 12/24/20, 6:07 PM:
-
Some assumptions do not apply which I have made. Resets are properly performed.
I am still rejecting the PR after rereading all comments and our codebsae and
will explain why:
* First of all {{TransferFailedExceptions}} wrap:
{code}
catch ( IOException | HttpException | InterruptedException e )
{
fireTransferError( resource, e, TransferEvent.REQUEST_PUT );
throw new TransferFailedException( formatTransferFailedMessage(
url, getProxyInfo() ), e );
}
{code}
Not all I/O exceptions can be recovered. {{HttpException}} is so generic that
most of the subtypes cannot be recovered from. This is not a way to go.
Especially not on such a high level because the changeset in the PR applies to
all stream wagons. Basically [~olegk] has already provide the way to go: One
needs to utilize
{{org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.createRetryHandler()}}
inside
{{org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(int,
InputData)}} to make work what you have in mind. That is something I would
really consider to include when there is a test depicting that it works. This
would not require to introduce any new user visiable changes.
was (Author: michael-o):
Some assumptions do not apply which I have made. Resets are properly performed.
I am still rejecting the PR after rereading all comments and our codebsae and
will explain why:
* First of all {{TransferFailedException}}s wrap:
{code}
catch ( IOException | HttpException | InterruptedException e )
{
fireTransferError( resource, e, TransferEvent.REQUEST_PUT );
throw new TransferFailedException( formatTransferFailedMessage(
url, getProxyInfo() ), e );
}
{code}
Not all I/O exceptions can be recovered. {{HttpException}} is so generic that
most of the subtypes cannot be recovered from. This is not a way to go.
Especially not on such a high level because the changeset in the PR applies to
all stream wagons. Basically [~olegk] has already provide the way to go: One
needs to utilize
{{org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.createRetryHandler()}}
inside
{{org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(int,
InputData)}} to make work what you have in mind. That is something I would
really consider to include when there is a test depicting that it works. This
would not require to introduce any new user visiable changes.
> Retry for connection issues
> ---
>
> Key: WAGON-575
> URL: https://issues.apache.org/jira/browse/WAGON-575
> Project: Maven Wagon
> Issue Type: New Feature
> Components: wagon-http
>Affects Versions: 3.3.4
> Environment: windows and linux
>Reporter: Michael Medin
>Priority: Minor
>
> There are a RetryHandler and now also a ServiceRetryHandler but both seem to
> focus only the handshake and no the data stream.
> In our case we download large artifacts (1+Gb) over a sometimes shaky
> connection which causes frequent "Connection reset" issues.
> To mitigate this we started to implement retry logic based on this
> [https://maven.apache.org/wagon/wagon-providers/wagon-http/] document. But
> seems our retry handler is never invoked when the connection is reset.
> So after some digging into the source code it seems the retryhandler are only
> used when connecting to the server and once the connection has been
> established and the HTTP headers have been read there is no retry handling
> for the reminder stream copy.
> Looking at the code it seems non trivial to add retry for this at is split up
> so I wanted to know if I am missing something before looking at implementing
> a PR for this.
>
> A simple way to simulate this behavior is to start a maven build with some
> large dependencies and during the download phase just kill the internet
> connection.
> If you have a RetryHandler enabled it will never be called instead you will
> get a stack trace along the following:
> {{[ERROR] Failed to execute goal on project test: Could not resolve
> dependencies for project test:test:jar:1.0.0: Could not transfer artifact
> REDACTED from/to REDACTED from REDACTED failed: Connection reset -> [Help 1]}}
> {{org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project test: Could not resolve dependencies for project
> test:test:jar:1.0.0: Could not transfer artifact REDACTED from/to REDACTED
> (REDACTED): GET request of: REDACTED from REDACTED failed}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDe
[jira] [Comment Edited] (WAGON-575) Retry for connection issues
[
https://issues.apache.org/jira/browse/WAGON-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245396#comment-17245396
]
Michael Osipov edited comment on WAGON-575 at 12/11/20, 6:13 PM:
-
Some assumptions do not apply which I have made. Resets are properly performed.
I am still rejecting the PR after rereading all comments and our codebsae and
will explain why:
* First of all {{TransferFailedException}}s wrap:
{code}
catch ( IOException | HttpException | InterruptedException e )
{
fireTransferError( resource, e, TransferEvent.REQUEST_PUT );
throw new TransferFailedException( formatTransferFailedMessage(
url, getProxyInfo() ), e );
}
{code}
Not all I/O exceptions can be recovered. {{HttpException}} is so generic that
most of the subtypes cannot be recovered from. This is not a way to go.
Especially not on such a high level because the changeset in the PR applies to
all stream wagons. Basically [~olegk] has already provide the way to go: One
needs to utilize
{{org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.createRetryHandler()}}
inside
{{org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(int,
InputData)}} to make work what you have in mind. That is something I would
really consider to include when there is a test depicting that it works. This
would not require to introduce any new user visiable changes.
was (Author: michael-o):
Some assumptions do not apply which I have made. Resets are properly performed.
I am still rejecting the PR after rereading all comments and our codebsae and
will explain why:
* First of all {{TransferFailedException}} wraps:
{code}
catch ( IOException | HttpException | InterruptedException e )
{
fireTransferError( resource, e, TransferEvent.REQUEST_PUT );
throw new TransferFailedException( formatTransferFailedMessage(
url, getProxyInfo() ), e );
}
{code}
Not all I/O exceptions can be recovered. {{HttpException}} is so generic that
most of the subtypes cannot be recovered from. This is not a way to go.
Especially not on such a high level because the changeset in the PR applies to
all stream wagons. Basically [~olegk] has already provide the way to go. One
needs to utilize
{{org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.createRetryHandler()}}
inside
{{org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(int,
InputData)}} to make work what you have in mind. That is something I would
really consider to include when there is a test depicting that it works. This
would not require to introduce any new user visiable changes.
> Retry for connection issues
> ---
>
> Key: WAGON-575
> URL: https://issues.apache.org/jira/browse/WAGON-575
> Project: Maven Wagon
> Issue Type: New Feature
> Components: wagon-http
>Affects Versions: 3.3.4
> Environment: windows and linux
>Reporter: Michael Medin
>Priority: Minor
>
> There are a RetryHandler and now also a ServiceRetryHandler but both seem to
> focus only the handshake and no the data stream.
> In our case we download large artifacts (1+Gb) over a sometimes shaky
> connection which causes frequent "Connection reset" issues.
> To mitigate this we started to implement retry logic based on this
> [https://maven.apache.org/wagon/wagon-providers/wagon-http/] document. But
> seems our retry handler is never invoked when the connection is reset.
> So after some digging into the source code it seems the retryhandler are only
> used when connecting to the server and once the connection has been
> established and the HTTP headers have been read there is no retry handling
> for the reminder stream copy.
> Looking at the code it seems non trivial to add retry for this at is split up
> so I wanted to know if I am missing something before looking at implementing
> a PR for this.
>
> A simple way to simulate this behavior is to start a maven build with some
> large dependencies and during the download phase just kill the internet
> connection.
> If you have a RetryHandler enabled it will never be called instead you will
> get a stack trace along the following:
> {{[ERROR] Failed to execute goal on project test: Could not resolve
> dependencies for project test:test:jar:1.0.0: Could not transfer artifact
> REDACTED from/to REDACTED from REDACTED failed: Connection reset -> [Help 1]}}
> {{org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project test: Could not resolve dependencies for project
> test:test:jar:1.0.0: Could not transfer artifact REDACTED from/to REDACTED
> (REDACTED): GET request of: REDACTED from REDACTED failed}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDe
[jira] [Comment Edited] (WAGON-575) Retry for connection issues
[
https://issues.apache.org/jira/browse/WAGON-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245321#comment-17245321
]
Igor Dvorzhak edited comment on WAGON-575 at 12/7/20, 4:19 PM:
---
I can add configuration if retry mechanism is acceptable in general.
It retries only `TransferFailedException` exceptions. I expect that wrong
credentials are covered by `AuthorizationException` exceptions that are not
retried.
I think that this PR does not need to deal with `InputStream` and progress
tracking because it retries whole file download e2e?
I agree that retries on a lower level will be a better and more efficient
solution, but from discussion in this thread I understood that nobody has a
time to implement them, so I proposed a quick fix that solves this problem.
was (Author: medb):
I can add configuration if retry mechanism is acceptable in general.
It retries only `TransferFailedException` exceptions. I expect that wrong
credentials are covered by `AuthorizationException` exceptions that are not
retried.
These PR is not dealing with `InputStream` and progress tracking because it
retries whole file download e2e.
I agree that retries on a lower level will be a better and more efficient
solution, but from discussion in this thread I understood that nobody has a
time to implement them, so I proposed a quick fix that solves this problem.
> Retry for connection issues
> ---
>
> Key: WAGON-575
> URL: https://issues.apache.org/jira/browse/WAGON-575
> Project: Maven Wagon
> Issue Type: New Feature
> Components: wagon-http
>Affects Versions: 3.3.4
> Environment: windows and linux
>Reporter: Michael Medin
>Priority: Minor
>
> There are a RetryHandler and now also a ServiceRetryHandler but both seem to
> focus only the handshake and no the data stream.
> In our case we download large artifacts (1+Gb) over a sometimes shaky
> connection which causes frequent "Connection reset" issues.
> To mitigate this we started to implement retry logic based on this
> [https://maven.apache.org/wagon/wagon-providers/wagon-http/] document. But
> seems our retry handler is never invoked when the connection is reset.
> So after some digging into the source code it seems the retryhandler are only
> used when connecting to the server and once the connection has been
> established and the HTTP headers have been read there is no retry handling
> for the reminder stream copy.
> Looking at the code it seems non trivial to add retry for this at is split up
> so I wanted to know if I am missing something before looking at implementing
> a PR for this.
>
> A simple way to simulate this behavior is to start a maven build with some
> large dependencies and during the download phase just kill the internet
> connection.
> If you have a RetryHandler enabled it will never be called instead you will
> get a stack trace along the following:
> {{[ERROR] Failed to execute goal on project test: Could not resolve
> dependencies for project test:test:jar:1.0.0: Could not transfer artifact
> REDACTED from/to REDACTED from REDACTED failed: Connection reset -> [Help 1]}}
> {{org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project test: Could not resolve dependencies for project
> test:test:jar:1.0.0: Could not transfer artifact REDACTED from/to REDACTED
> (REDACTED): GET request of: REDACTED from REDACTED failed}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
> (LifecycleDependencyResolver.java:269)}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
> (LifecycleDependencyResolver.java:147)}}
> {{ at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
> (MojoExecutor.java:248)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:202)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)}}
> {{ at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)}}
> {{ at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:1
[jira] [Comment Edited] (WAGON-575) Retry for connection issues
[
https://issues.apache.org/jira/browse/WAGON-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244923#comment-17244923
]
Igor Dvorzhak edited comment on WAGON-575 at 12/7/20, 3:22 AM:
---
I have sent a [PR|https://github.com/apache/maven-wagon/pull/75] that fixes
this issue by adding high-level retries for failed downloads in Maven Wagon.
It's less efficient retry mechanism than what was discussed in the comments
above but much simpler to implement and solves stability issue with my
multi-hour build.
Please, let me know if this is something that you are willing to consider/merge
with some additional work (configuration, tests, etc).
was (Author: medb):
I have sent a [PR|https://github.com/apache/maven-wagon/pull/75] that fixes
this issue by adding high-level retries for failed downloads in Maven Wagon.
It's less efficient retry mechanism than what was discussed in the comments
above but much simpler to implement and solves the issue for my build.
Please, let me know if this is something that you are willing to consider/merge
with some additional work (configuration, tests, etc).
> Retry for connection issues
> ---
>
> Key: WAGON-575
> URL: https://issues.apache.org/jira/browse/WAGON-575
> Project: Maven Wagon
> Issue Type: New Feature
> Components: wagon-http
>Affects Versions: 3.3.4
> Environment: windows and linux
>Reporter: Michael Medin
>Priority: Minor
>
> There are a RetryHandler and now also a ServiceRetryHandler but both seem to
> focus only the handshake and no the data stream.
> In our case we download large artifacts (1+Gb) over a sometimes shaky
> connection which causes frequent "Connection reset" issues.
> To mitigate this we started to implement retry logic based on this
> [https://maven.apache.org/wagon/wagon-providers/wagon-http/] document. But
> seems our retry handler is never invoked when the connection is reset.
> So after some digging into the source code it seems the retryhandler are only
> used when connecting to the server and once the connection has been
> established and the HTTP headers have been read there is no retry handling
> for the reminder stream copy.
> Looking at the code it seems non trivial to add retry for this at is split up
> so I wanted to know if I am missing something before looking at implementing
> a PR for this.
>
> A simple way to simulate this behavior is to start a maven build with some
> large dependencies and during the download phase just kill the internet
> connection.
> If you have a RetryHandler enabled it will never be called instead you will
> get a stack trace along the following:
> {{[ERROR] Failed to execute goal on project test: Could not resolve
> dependencies for project test:test:jar:1.0.0: Could not transfer artifact
> REDACTED from/to REDACTED from REDACTED failed: Connection reset -> [Help 1]}}
> {{org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project test: Could not resolve dependencies for project
> test:test:jar:1.0.0: Could not transfer artifact REDACTED from/to REDACTED
> (REDACTED): GET request of: REDACTED from REDACTED failed}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
> (LifecycleDependencyResolver.java:269)}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
> (LifecycleDependencyResolver.java:147)}}
> {{ at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
> (MojoExecutor.java:248)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:202)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)}}
> {{ at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)}}
> {{ at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)}}
> {{ at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)}}
> {{ at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)}}
> {{ at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)}}
> {{ at jdk.internal.reflect.NativeMethodAccessorImpl.in
[jira] [Comment Edited] (WAGON-575) Retry for connection issues
[
https://issues.apache.org/jira/browse/WAGON-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244923#comment-17244923
]
Igor Dvorzhak edited comment on WAGON-575 at 12/7/20, 3:21 AM:
---
I have sent a [PR|https://github.com/apache/maven-wagon/pull/75] that fixes
this issue by adding high-level retries for failed downloads in Maven Wagon.
It's less efficient retry mechanism than what was discussed in the comments
above but much simpler to implement and solves the issue for my build.
Please, let me know if this is something that you are willing to consider/merge
with some additional work (configuration, tests, etc).
was (Author: medb):
I have sent a [PR|https://github.com/apache/maven-wagon/pull/75] that fixes
this issue by adding high-level retries for failed downloads. It's less
efficient retry mechanism than what was discussed in the comments above but
much simpler to implement and solves the issue for my build.
Please, let me know if this is something that you are willing to consider/merge
with some additional work (configuration, tests, etc).
> Retry for connection issues
> ---
>
> Key: WAGON-575
> URL: https://issues.apache.org/jira/browse/WAGON-575
> Project: Maven Wagon
> Issue Type: New Feature
> Components: wagon-http
>Affects Versions: 3.3.4
> Environment: windows and linux
>Reporter: Michael Medin
>Priority: Minor
>
> There are a RetryHandler and now also a ServiceRetryHandler but both seem to
> focus only the handshake and no the data stream.
> In our case we download large artifacts (1+Gb) over a sometimes shaky
> connection which causes frequent "Connection reset" issues.
> To mitigate this we started to implement retry logic based on this
> [https://maven.apache.org/wagon/wagon-providers/wagon-http/] document. But
> seems our retry handler is never invoked when the connection is reset.
> So after some digging into the source code it seems the retryhandler are only
> used when connecting to the server and once the connection has been
> established and the HTTP headers have been read there is no retry handling
> for the reminder stream copy.
> Looking at the code it seems non trivial to add retry for this at is split up
> so I wanted to know if I am missing something before looking at implementing
> a PR for this.
>
> A simple way to simulate this behavior is to start a maven build with some
> large dependencies and during the download phase just kill the internet
> connection.
> If you have a RetryHandler enabled it will never be called instead you will
> get a stack trace along the following:
> {{[ERROR] Failed to execute goal on project test: Could not resolve
> dependencies for project test:test:jar:1.0.0: Could not transfer artifact
> REDACTED from/to REDACTED from REDACTED failed: Connection reset -> [Help 1]}}
> {{org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project test: Could not resolve dependencies for project
> test:test:jar:1.0.0: Could not transfer artifact REDACTED from/to REDACTED
> (REDACTED): GET request of: REDACTED from REDACTED failed}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
> (LifecycleDependencyResolver.java:269)}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
> (LifecycleDependencyResolver.java:147)}}
> {{ at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
> (MojoExecutor.java:248)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:202)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)}}
> {{ at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)}}
> {{ at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)}}
> {{ at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)}}
> {{ at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)}}
> {{ at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)}}
> {{ at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)}}
> {{ at jd
[jira] [Comment Edited] (WAGON-575) Retry for connection issues
[
https://issues.apache.org/jira/browse/WAGON-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17042738#comment-17042738
]
Michael Osipov edited comment on WAGON-575 at 2/22/20 10:55 PM:
That is going to require
* the server to accept ranges
* us to memorize that
* resume the download
* simulate a TCP reset for this to test. I have no idea how to do that in Java
for now.
and likely other stuff. While this sounds like a very interesting task it takes
a serious amount of time to implement that.
For an average JAR of a few hundred kilobytes this is overkill.
was (Author: michael-o):
That is going to require
* the server to accept ranges
* us to memorize that
* resume the download
* simulate a TCP reset for this to test. I have no idea how to do that in Java
for now.
and likely other stuff. While this sounds like a very interesting task it takes
a serious amount of time to implement that.
> Retry for connection issues
> ---
>
> Key: WAGON-575
> URL: https://issues.apache.org/jira/browse/WAGON-575
> Project: Maven Wagon
> Issue Type: New Feature
> Components: wagon-http
>Affects Versions: 3.3.4
> Environment: windows and linux
>Reporter: Michael Medin
>Priority: Minor
>
> There are a RetryHandler and now also a ServiceRetryHandler but both seem to
> focus only the handshake and no the data stream.
> In our case we download large artifacts (1+Gb) over a sometimes shaky
> connection which causes frequent "Connection reset" issues.
> To mitigate this we started to implement retry logic based on this
> [https://maven.apache.org/wagon/wagon-providers/wagon-http/] document. But
> seems our retry handler is never invoked when the connection is reset.
> So after some digging into the source code it seems the retryhandler are only
> used when connecting to the server and once the connection has been
> established and the HTTP headers have been read there is no retry handling
> for the reminder stream copy.
> Looking at the code it seems non trivial to add retry for this at is split up
> so I wanted to know if I am missing something before looking at implementing
> a PR for this.
>
> A simple way to simulate this behavior is to start a maven build with some
> large dependencies and during the download phase just kill the internet
> connection.
> If you have a RetryHandler enabled it will never be called instead you will
> get a stack trace along the following:
> {{[ERROR] Failed to execute goal on project test: Could not resolve
> dependencies for project test:test:jar:1.0.0: Could not transfer artifact
> REDACTED from/to REDACTED from REDACTED failed: Connection reset -> [Help 1]}}
> {{org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project test: Could not resolve dependencies for project
> test:test:jar:1.0.0: Could not transfer artifact REDACTED from/to REDACTED
> (REDACTED): GET request of: REDACTED from REDACTED failed}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
> (LifecycleDependencyResolver.java:269)}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
> (LifecycleDependencyResolver.java:147)}}
> {{ at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
> (MojoExecutor.java:248)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:202)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)}}
> {{ at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)}}
> {{ at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)}}
> {{ at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)}}
> {{ at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)}}
> {{ at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)}}
> {{ at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)}}
> {{ at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)}}
> {{ at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (Del
[jira] [Comment Edited] (WAGON-575) Retry for connection issues
[
https://issues.apache.org/jira/browse/WAGON-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17042738#comment-17042738
]
Michael Osipov edited comment on WAGON-575 at 2/22/20 10:51 PM:
That is going to require
* the server to accept ranges
* us to memorize that
* resume the download
* simulate a TCP reset for this to test. I have no idea how to do that in Java
for now.
and likely other stuff. While this sounds like a very interesting task it takes
a serious amount of time to implement that.
was (Author: michael-o):
That is going to require
* the server to accept ranges
* us to memorize that
* resume the download
and likely other stuff. While this sounds like a very interesting task it takes
a serious amount of time to implement that.
> Retry for connection issues
> ---
>
> Key: WAGON-575
> URL: https://issues.apache.org/jira/browse/WAGON-575
> Project: Maven Wagon
> Issue Type: New Feature
> Components: wagon-http
>Affects Versions: 3.3.4
> Environment: windows and linux
>Reporter: Michael Medin
>Priority: Minor
>
> There are a RetryHandler and now also a ServiceRetryHandler but both seem to
> focus only the handshake and no the data stream.
> In our case we download large artifacts (1+Gb) over a sometimes shaky
> connection which causes frequent "Connection reset" issues.
> To mitigate this we started to implement retry logic based on this
> [https://maven.apache.org/wagon/wagon-providers/wagon-http/] document. But
> seems our retry handler is never invoked when the connection is reset.
> So after some digging into the source code it seems the retryhandler are only
> used when connecting to the server and once the connection has been
> established and the HTTP headers have been read there is no retry handling
> for the reminder stream copy.
> Looking at the code it seems non trivial to add retry for this at is split up
> so I wanted to know if I am missing something before looking at implementing
> a PR for this.
>
> A simple way to simulate this behavior is to start a maven build with some
> large dependencies and during the download phase just kill the internet
> connection.
> If you have a RetryHandler enabled it will never be called instead you will
> get a stack trace along the following:
> {{[ERROR] Failed to execute goal on project test: Could not resolve
> dependencies for project test:test:jar:1.0.0: Could not transfer artifact
> REDACTED from/to REDACTED from REDACTED failed: Connection reset -> [Help 1]}}
> {{org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project test: Could not resolve dependencies for project
> test:test:jar:1.0.0: Could not transfer artifact REDACTED from/to REDACTED
> (REDACTED): GET request of: REDACTED from REDACTED failed}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
> (LifecycleDependencyResolver.java:269)}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
> (LifecycleDependencyResolver.java:147)}}
> {{ at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
> (MojoExecutor.java:248)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:202)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)}}
> {{ at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)}}
> {{ at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)}}
> {{ at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)}}
> {{ at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)}}
> {{ at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)}}
> {{ at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)}}
> {{ at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)}}
> {{ at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)}}
> {{ at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)}}
> {{ at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke (Method.java:566)}}
> {{ at org.codehaus.plexus.classworlds.launcher.Launcher
[jira] [Comment Edited] (WAGON-575) Retry for connection issues
[
https://issues.apache.org/jira/browse/WAGON-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024323#comment-17024323
]
Michael Osipov edited comment on WAGON-575 at 1/27/20 1:06 PM:
---
bq. Does this mean retry for connection issue is beyond the scope of maven,
wagon, http client or all of them?
As mentioned by Oleg, not at all, but not in HttpClient. This would require
HttpClient to pipe through the input stream, catch exceptions and handle them
for you. This is best handled in client code. This has to be implemented in
Wagon. It is not that easy, these things have to be considered:
* Shall this apply for GET only? Or all idempotent requests?
* Wagon triggers events on state change, this needs to added/handled
* New properties for a retry count have to be added
* Count needs to be maintained with new methods
* How to test this? One needs to send a TCP RST, I was manage to do that by
killing the Tomcat process. Does not seem to be feasible in unit tests.
Unless someone makes a proposal, this will have a low priority for me since I
mostly focus on bugs in Wagon.
was (Author: michael-o):
bq. Does this mean retry for connection issue is beyond the scope of maven,
wagon, http client or all of them?
As mentioned by Oleg, not all all, but not in HttpClient. This would require
HttpClient to pipe through the input stream, catch exceptions and handle them
for your. This is best handled in client code. This has to be implemented in
Wagon. It is not that easy, these things have to be observed:
* Shall this apply for GET only? Or all idempotent requests?
* Wagon triggers events on state change, this needs to added
* New properties for a retry count have to be added
* Counts needs to be maintained with new methods
* How to test this? One needs to send a TCP RST, I was manage to do that by
killing the Tomcat process. Does not seems to be feasible in unit tests.
Unless someone makes a proposal, this will have a low priority for me since I
mostly focus on bugs in Wagon.
> Retry for connection issues
> ---
>
> Key: WAGON-575
> URL: https://issues.apache.org/jira/browse/WAGON-575
> Project: Maven Wagon
> Issue Type: Improvement
> Components: wagon-http
>Affects Versions: 3.3.4
> Environment: windows and linux
>Reporter: Michael Medin
>Priority: Major
>
> There are a RetryHandler and now also a ServiceRetryHandler but both seem to
> focus only the handshake and no the data stream.
> In our case we download large artifacts (1+Gb) over a sometimes shaky
> connection which causes frequent "Connection reset" issues.
> To mitigate this we started to implement retry logic based on this
> [https://maven.apache.org/wagon/wagon-providers/wagon-http/] document. But
> seems our retry handler is never invoked when the connection is reset.
> So after some digging into the source code it seems the retryhandler are only
> used when connecting to the server and once the connection has been
> established and the HTTP headers have been read there is no retry handling
> for the reminder stream copy.
> Looking at the code it seems non trivial to add retry for this at is split up
> so I wanted to know if I am missing something before looking at implementing
> a PR for this.
>
> A simple way to simulate this behavior is to start a maven build with some
> large dependencies and during the download phase just kill the internet
> connection.
> If you have a RetryHandler enabled it will never be called instead you will
> get a stack trace along the following:
> {{[ERROR] Failed to execute goal on project test: Could not resolve
> dependencies for project test:test:jar:1.0.0: Could not transfer artifact
> REDACTED from/to REDACTED from REDACTED failed: Connection reset -> [Help 1]}}
> {{org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project test: Could not resolve dependencies for project
> test:test:jar:1.0.0: Could not transfer artifact REDACTED from/to REDACTED
> (REDACTED): GET request of: REDACTED from REDACTED failed}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies
> (LifecycleDependencyResolver.java:269)}}
> {{ at
> org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies
> (LifecycleDependencyResolver.java:147)}}
> {{ at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved
> (MojoExecutor.java:248)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:202)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)}}
> {{ at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)}}
> {{ at org.apache.maven.lifecycle.internal.Life
