Re: [Bacula-users] Troubles with AWS [Possibly solved]

2023-01-16 Thread Chris Wilkinson
I don't specify "max concurrent uploads" so it defaults to 3 per manual. I
have differing max part size for the jobs but the ones that frequently fail
are 500Mb.

I'm bandwidth limited to 6MB/sec so I don't see any benefit from trying to
run uploads concurrently. My largest backup is 350GB that takes around 20h.

Let us know what values seem to work best.

Chris


On Mon, 16 Jan 2023, 8:05 am Andrea Venturoli,  wrote:

> On 1/13/23 18:26, Andrea Venturoli wrote:
>
> > Then I upgraded every package (don't think it matters, still no harm)
> > and now I'm trying with "cloud upload" manually. I'll see how this goes.
>
> Still troublesome, but now the error is more verbose:
>
> > 3000 Cloud Upload: Full0116/part.195   state=error   size=99.99 MB
> duration=19871s msg=S3_put_object ERR=RequestTorrentOfBucketError CURL
> Effective URL: https://s3.eu-south-1.amazonaws.com/Full0116/part.195 CURL
> OS Error: 54 CURL Effective URL:
> https://s3.eu-south-1.amazonaws.com/Full0116/part.195 CURL OS Error: 54
> retry=10
>
> So we get "CURL OS Error: 54".
>
> If I get it right errno=54 is "ECONNRESET Connection reset by peer. A
> connection was forcibly closed by a peer.  This normally results from a
> loss of the connection on the remote socket due to a timeout or a reboot".
>
>
>
>
> So I tried lowering "Maximum Concurrent Uploads" from 20 to 1.
> Now it work properly, although uploading a full backup will take some 2
> days...
> I'll try fiddling again with with intermediate values.
>
> Another thing I though of would be to lower "maximum part size",
> currently at 100MB. Of course this would not work on the already taken
> backup, I think.
>
>   bye & Thanks
> av.
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Troubles with AWS [Possibly solved]

2023-01-16 Thread Andrea Venturoli

On 1/13/23 18:26, Andrea Venturoli wrote:

Then I upgraded every package (don't think it matters, still no harm) 
and now I'm trying with "cloud upload" manually. I'll see how this goes.


Still troublesome, but now the error is more verbose:


3000 Cloud Upload: Full0116/part.195   state=error   size=99.99 MB 
duration=19871s msg=S3_put_object ERR=RequestTorrentOfBucketError CURL 
Effective URL: https://s3.eu-south-1.amazonaws.com/Full0116/part.195 CURL OS 
Error: 54 CURL Effective URL: 
https://s3.eu-south-1.amazonaws.com/Full0116/part.195 CURL OS Error: 54  
retry=10


So we get "CURL OS Error: 54".

If I get it right errno=54 is "ECONNRESET Connection reset by peer. A 
connection was forcibly closed by a peer.  This normally results from a 
loss of the connection on the remote socket due to a timeout or a reboot".





So I tried lowering "Maximum Concurrent Uploads" from 20 to 1.
Now it work properly, although uploading a full backup will take some 2 
days...

I'll try fiddling again with with intermediate values.

Another thing I though of would be to lower "maximum part size", 
currently at 100MB. Of course this would not work on the already taken 
backup, I think.


 bye & Thanks
av.


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users