Re: [Bacula-users] bacula-sd can't upload to AWS S3
Hello, śr., 22 wrz 2021 o 21:14 Otaci Martins napisał(a): > SOLVED! > > It was a proxy problem. Necessary to include a line in systemd > bacula-sd.service: > ... > [Service] > Type=simple > User=root > Group=disk > Environment="https_proxy=http://:" > ExecStart=/opt/bacula/bin/bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf > SuccessExitStatus=15 > LimitMEMLOCK=infinity > ... > > When running bacula-sd from login shell command line it uses http_proxy > yet defined in the environment, this is because things go well. > I'm glad you solved it! best regards Radek -- Radosław Korzeniewski rados...@korzeniewski.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-sd can't upload to AWS S3
SOLVED!
It was a proxy problem. Necessary to include a line in systemd
bacula-sd.service:
...
[Service]
Type=simple
User=root
Group=disk
Environment="https_proxy=http://:"
ExecStart=/opt/bacula/bin/bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf
SuccessExitStatus=15
LimitMEMLOCK=infinity
...
When running bacula-sd from login shell command line it uses http_proxy yet
defined in the environment, this is because things go well.
Unfortunately, bacula-sd debug (level 500 or more) did not show any clue
about the problem' s cause.
Thanks to all for the assistance.
Otaci Martins
Em seg., 20 de set. de 2021 às 16:27, Otaci Martins
escreveu:
> bacula-sd can't upload volumes to AWS S3 if running as systemd daemon, but
> if called directly by terminal cli the upload works perfectly.
>
> 1. Steps to reproduce the upload error:
> ---
> $ sudo systemctl start bacula-sd
>
> $ ps -ef | grep bacula
> root 12573 1 0 15:39 ?00:00:00 /opt/bacula/bin/bacula-sd
> -fP -c /opt/bacula/etc/bacula-sd.conf
> root 13555 1 0 Sep15 ?00:00:01 /opt/bacula/bin/bacula-fd
> -fP -c /opt/bacula/etc/bacula-fd.conf
> bacula 29641 1 0 10:58 ?00:00:00 /opt/bacula/bin/bacula-dir
> -fP -c /opt/bacula/etc/bacula-dir.conf
>
> $ echo "cloud list storage=S3CLOUD" | bconsole
> Connecting to Director 192.168.10.12:9101
> 1000 OK: 10002 BACULA11-DIR Version: 11.0.5 (03 June 2021)
> Enter a period to cancel a command.
> cloud list storage=S3CLOUD
> Automatically selected Catalog: BACULA11-CA
> Using Catalog "BACULA11-CA"
> Connecting to Storage daemon S3CLOUD at 192.168.10.12:9103 ...
> 3900 Error cannot get cloud Volume list. ERR=S3_list_bucket
> ERR=FailedToConnect CURL Effective URL:
> https://mybucket.s3.sa-east-1.amazonaws.com/?delimiter=%2F CURL OS Error:
> 110 CURL Effective URL:
> https://mybucket.s3.sa-east-1.amazonaws.com/?delimiter=%2F CURL OS Error:
> 110
>
> 2. Steps to reproduce the upload success:
> -
> $ sudo systemctl stop bacula-sd
>
> $ sudo /opt/bacula/bin/bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf
>
> $ ps -ef|grep bacula
> root 13362 11959 0 15:53 pts/100:00:00 sudo
> /opt/bacula/bin/bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf
> root 13364 13362 0 15:53 pts/100:00:00 /opt/bacula/bin/bacula-sd
> -fP -c /opt/bacula/etc/bacula-sd.conf
> root 13555 1 0 Sep15 ?00:00:01 /opt/bacula/bin/bacula-fd
> -fP -c /opt/bacula/etc/bacula-fd.conf
> bacula 29641 1 0 10:58 ?00:00:00 /opt/bacula/bin/bacula-dir
> -fP -c /opt/bacula/etc/bacula-dir.conf
>
> $ echo "cloud list storage=S3CLOUD" | bconsole
> Connecting to Director 192.168.10.12:9101
> 1000 OK: 10002 BACULA11-DIR Version: 11.0.5 (03 June 2021)
> Enter a period to cancel a command.
> cloud list storage=S3CLOUD
> Automatically selected Catalog: BACULA11-CA
> Using Catalog "BACULA11-CA"
> Connecting to Storage daemon S3CLOUD at 192.168.10.12:9103 ...
>
> ++---+--+--+---+
> |Volume Name | Status | Media Type | Pool
> |VolBytes |
>
> ++---+--+--+---+
> | AWSS3-0076 | Full | CloudAWS |
> AWSS3-PO | 8.528 MB |
> |BACKUP-CATALOGO |Append | CloudAWS |
> AWSS3-PO | 1.015 MB |
>
> ++---+--+--+---+
>
> 3. My question:
> ---
> So, once the config files are exactly the same in the two scenarios, what
> can i search for?
>
> 4. My specs:
>
> Bacula CE 11.0.5 (installed from official rpm repos)
>
> SO:
>
> CentOS Linux release 7.9.2009 (Core)
> Linux 3.10.0-1160.15.2.el7.x86_64 #1 SMP Wed Feb 3 15:06:38 UTC 2021
> x86_64 x86_64 x86_64 GNU/Linux
>
> bacula-dir.conf:
>
> Storage {
> Name = S3CLOUD
> Address = 192.168.10.12
> SDPort = 9103
> Password = "bacula"
> Device = AWSS3-DRIVE-01
> Media Type = CloudAWS
> Maximum Concurrent Jobs = 10
> }
>
> bacula-sd.conf:
>
> Device {
> Name = AWSS3-DRIVE-01
> Device Type = Cloud
> Cloud = AWSS3
> Archive Device = /tmp/hdd03
> Maximum Part Size = 1 GB
> Maximum File Size = 1 GB
> Media Type = CloudAWS
> LabelMedia = yes
> Random Access = yes
> AutomaticMount = yes
> RemovableMedia = no
> AlwaysOpen = no
> }
>
> Cloud {
> Name = AWSS3
> Driver = "S3"
> HostName = "s3.sa-east-1.amazonaws.com"
> BucketName = "MyBucket"
> AccessKey = "MyAccessKey"
> SecretKey = "MySecretKey"
> Protocol = HTTPS
> UriStyle = VirtualHost
> Truncate Cache = No
> Upload = EachPart
> Region = "sa-east-1"
> MaximumUploadBandwidth = 5MB/s
> }
>
> Systemd bacula-sd.service:
>
> [Unit]
> Description=Bacula Storage Daemon service
> Requires=network.target
> After=network.target
> RequiresMountsFor=/opt/bac
Re: [Bacula-users] bacula-sd can't upload to AWS S3
My mistake in last message. Here is the correct text: ... The fact is that the only difference is how bacula-sd is invoked: 1. $ sudo systemctl start bacula-sd => UPLOAD ERROR 2. $ sudo bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf => UPLOAD OK Best regards. Otaci MArtins Em ter., 21 de set. de 2021 às 11:17, Otaci Martins escreveu: > Heitor, this is only a LAB not a production system. > > First i am using user=root, but after your last message i changed to > user=bacula with same results. The cache /tmp in this LAB does not affect > the results here. > > The fact is that the only difference is how bacula-sd is invoked: > >1. $ sudo systemctl start bacula-sd => UPLOAD OK >2. $ sudo bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf => UPLOAD >ERROR > > All other configurations and environments are exactly the same, running in > the same machine, just changing how bacula-sd is invoked. > > Tks > Otaci Martins > > Em ter., 21 de set. de 2021 às 10:56, Heitor Faria > escreveu: > >> Hello Otaci, >> >> IMHO your scenario is kind of confusing. >> >> 1. Your service init file is using the root Linux user. Like I wrote >> before, not recommendable: >> "[Service] >> Type=simple >> User=root" >> >> 2. You are using /tmp as cache/volume storage, what is absolutely not >> recommendable. >> >> Regards, >> > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-sd can't upload to AWS S3
Heitor, this is only a LAB not a production system. First i am using user=root, but after your last message i changed to user=bacula with same results. The cache /tmp in this LAB does not affect the results here. The fact is that the only difference is how bacula-sd is invoked: 1. $ sudo systemctl start bacula-sd => UPLOAD OK 2. $ sudo bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf => UPLOAD ERROR All other configurations and environments are exactly the same, running in the same machine, just changing how bacula-sd is invoked. Tks Otaci Martins Em ter., 21 de set. de 2021 às 10:56, Heitor Faria escreveu: > Hello Otaci, > > IMHO your scenario is kind of confusing. > > 1. Your service init file is using the root Linux user. Like I wrote > before, not recommendable: > "[Service] > Type=simple > User=root" > > 2. You are using /tmp as cache/volume storage, what is absolutely not > recommendable. > > Regards, > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-sd can't upload to AWS S3
Hello Otaci, IMHO your scenario is kind of confusing. 1. Your service init file is using the root Linux user. Like I wrote before, not recommendable: "[Service] Type=simple User=root" 2. You are using /tmp as cache/volume storage, what is absolutely not recommendable. Regards, ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-sd can't upload to AWS S3
Yes, Heitor, i know bacula-sd must run with bacula user. But when i was debbuging with "sudo bacula-sd -fP -d 200", and the volumes were successfully uploaded, i guess if could be a matter of user permission and runned bacula-sd service with user root, but without success. So, here is the debug output when running bacula-sd from command line (upload OK): $ sudo bacula-sd -u bacula -fP -d 200 bacula-sd: address_conf.c:289-0 Initaddr 0.0.0.0:9103 LAB001PGSQL-SD: jcr.c:131-0 read_last_jobs seek to 192 LAB001PGSQL-SD: jcr.c:138-0 Read num_items=10 LAB001PGSQL-SD: plugins.c:97-0 load_plugins LAB001PGSQL-SD: plugins.c:133-0 Rejected plugin: want=*-sd.so got name=bpipe-fd.so len=11 LAB001PGSQL-SD: plugins.c:133-0 Rejected plugin: want=*-sd.so got name= bacula-sd-cloud-driver-11.0.5.so len=32 LAB001PGSQL-SD: plugins.c:133-0 Rejected plugin: want=*-sd.so got name= bacula-sd-cloud-s3-driver-11.0.5.so len=35 LAB001PGSQL-SD: plugins.c:121-0 Failed to find any plugins in /opt/bacula/plugins LAB001PGSQL-SD: stored.c:704-0 calling init_dev AWSS3-DRIVE-01 LAB001PGSQL-SD: init_dev.c:141-0 init_dev dev_type=14 LAB001PGSQL-SD: init_dev.c:177-0 Num drivers=15 LAB001PGSQL-SD: init_dev.c:190-0 loadable=1 type=14 loaded=0 name=cloud handle=0 LAB001PGSQL-SD: init_dev.c:476-0 loadable=1 type=14 loaded=0 name=cloud handle=0 LAB001PGSQL-SD: init_dev.c:481-0 Open SD driver at /opt/bacula/plugins/ bacula-sd-cloud-driver-11.0.5.so LAB001PGSQL-SD: init_dev.c:484-0 Driver=cloud handle=7ffadc00ba20 LAB001PGSQL-SD: init_dev.c:486-0 Lookup "BaculaSDdriver" in driver=cloud LAB001PGSQL-SD: init_dev.c:488-0 Driver=cloud entry point=7ffae8b024d0 LAB001PGSQL-SD: cloud_dev.c:876-0 Open SD driver at /opt/bacula/plugins/ bacula-sd-cloud-s3-driver-11.0.5.so LAB001PGSQL-SD: cloud_dev.c:879-0 Driver=s3 handle=7ffadc00cec0 LAB001PGSQL-SD: cloud_dev.c:881-0 Lookup "BaculaCloudDriver" in driver=s3 LAB001PGSQL-SD: cloud_dev.c:883-0 Driver=s3 entry point=7ffae88ead30 LAB001PGSQL-SD: htable.c:67-0 malloc buf=7ffaf47e4038 size=100 rem=76 LAB001PGSQL-SD: init_dev.c:235-0 init_dev allocated: 7ffadc00c048 LAB001PGSQL-SD: init_dev.c:437-0 init_dev: tape=0 dev_name=/tmp/hdd03 LAB001PGSQL-SD: dev.c:1143-0 DEVICE::register_metrics called. 0x7ffadc00c048 collector=0x1b24998 LAB001PGSQL-SD: stored.c:706-0 SD init done AWSS3-DRIVE-01 (0x7ffadc00c048) LAB001PGSQL-SD: block_util.c:174-0 New block adata=0 len=64512 block=7ffadc019f80 LAB001PGSQL-SD: acquire.c:671-0 Attach 0xdc019928 to dev "AWSS3-DRIVE-01" (/tmp/hdd03) LAB001PGSQL-SD: events.c:48-0 Events: code=SD0002 daemon=LAB001PGSQL-SD ref=0x238f type=daemon source=*Daemon* text=Device initialization AWSS3-DRIVE-01 LAB001PGSQL-SD: events.c:48-0 Events: code=SD0001 daemon=LAB001PGSQL-SD ref=0x238f type=daemon source=*Daemon* text=Storage startup LAB001PGSQL-SD: bnet_server.c:90-0 Addresses 0.0.0.0:9103 LAB001PGSQL-SD: bsock.c:861-0 socket=4 who=client host=192.168.10.12 port=9103<== After this point we can see output of "cloud list storage=S3CLOUD" command. LAB001PGSQL-SD: bnet_server.c:235-0 Accept socket=192.168.10.12.9103:192.168.10.12.47838 s=0x1b26ae8 LAB001PGSQL-SD: dircmd.c:196-0 Got a DIR connection at 21-Sep-2021 10:17:05 LAB001PGSQL-SD: authenticatebase.cc:365-0 TLSPSK Remote need 0 LAB001PGSQL-SD: authenticatebase.cc:335-0 TLSPSK Local need 0 LAB001PGSQL-SD: cram-md5.c:68-0 send: auth cram-md5 challenge <452667247.1632230225@LAB001PGSQL-SD> ssl=0 LAB001PGSQL-SD: cram-md5.c:132-0 cram-get received: auth cram-md5 <56019976.1632230225@BACULA11-DIR> ssl=0 LAB001PGSQL-SD: cram-md5.c:156-0 sending resp to challenge: c+/ec2dS+/wbd4/iDyVQuC LAB001PGSQL-SD: dircmd.c:227-0 Message channel init completed. LAB001PGSQL-SD: dircmd.c:234-0 ssl=0 Sep 21 10:26:07 LAB001PGSQL bacula-sd[2437]: LAB001PGSQL-SD: cram-md5.c:132-0 cram-get received: auth cram-md5 <934531061.1632230767@BACULA11-DIR> ssl=0 Sep 21 10:26:07 LAB001PGSQL bacula-sd[2437]: LAB001PGSQL-SD: cram-md5.c:156-0 sending resp to challenge: R8/qOj/1B7kh97wbg/ZhvB Sep 21 10:26:07 LAB001PGSQL bacula-sd[2437]: LAB001PGSQL-SD: dircmd.c:227-0 Message channel init completed. Sep 21 10:26:07 LAB001PGSQL bacula-sd[2437]: LAB001PGSQL-SD: dircmd.c:234-0 https://mybucket.s3.sa-east-1.amazonaws.com/?delimiter=%2F CURL OS Error: 110 CURL Effective URL: https://mybucket.s3.sa-east-1.amazonaws.com/?delimiter=%2F CURL OS Error: 110 Obs: mybucket replaces the real bucket name. --- As we can see, debug does not show differences in the two scenarios. Best regards. Otaci Martins Em ter., 21 de set. de 2021 às 09:05, Heitor Faria escreveu: > Otaci, > > "sudo /opt/bacula/bin/bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf" > > Please run the SD with the Linux always bacula user. Debug mode should > give you more information. E.g. > > "sudo -u bacula bacula-sd -d 200" > > Rgds. > -- > MSc Heitor Faria (Miami/USA) > CEO Bacula LatAm > mobile1: + 1 909 655-8971 > mobile2: + 55 61 98268-4220 > > América Latina > [ http://bacula.lat/] > __
Re: [Bacula-users] bacula-sd can't upload to AWS S3
Otaci, "sudo /opt/bacula/bin/bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf" Please run the SD with the Linux always bacula user. Debug mode should give you more information. E.g. "sudo -u bacula bacula-sd -d 200" Rgds. -- MSc Heitor Faria (Miami/USA) CEO Bacula LatAm mobile1: + 1 909 655-8971 mobile2: + 55 61 98268-4220 América Latina [ http://bacula.lat/]___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-sd can't upload to AWS S3
Hi Heitor, You got this because the real bucket name was changed to "mybucket" in order to hide it. Here is the response with the real bucket name: AccessDenied Access Denied 10DA27806ZQK31JP GwrZZV1ZKQ2GzZcpCyVUys/wQmZP21n3VLP7mkIdiIivWCZGh5mtOmnBr8/REmj7G/Yp3aLPJGM= So, why there is no access denied when running bacula-sd started directly from terminal command line? All other things are the same, the only difference between fail and success is the mode how to bacula-sd is started. Tks Otaci Martins Em ter., 21 de set. de 2021 às 08:43, Heitor Faria escreveu: > Hello Otaci, > > If you click in the error URL it will point you what's is wrong. I believe > you are using the wrong service AWS endpoint address: > > "This XML file does not appear to have any style information associated > with it. The document tree is shown below. > > > PermanentRedirect > The bucket you are attempting to access must be addressed using > the specified endpoint. Please send all future requests to this > endpoint. > s3.amazonaws.com > mybucket > KMXXEQJBMJVK965E > > z7u4pxRSwCHji+wl1sNqDAeDVvc3AHo6RqVbC7Ur5XvexPnCyhg0bfeNbqzDUjepYdYKt3iuGwE= > " > > Rgds. > -- > MSc Heitor Faria (Miami/USA) > CEO Bacula LatAm > mobile1: + 1 909 655-8971 > mobile2: + 55 61 98268-4220 > > América Latina > [ http://bacula.lat/] > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-sd can't upload to AWS S3
Hello Otaci, If you click in the error URL it will point you what's is wrong. I believe you are using the wrong service AWS endpoint address: "This XML file does not appear to have any style information associated with it. The document tree is shown below. PermanentRedirect The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. s3.amazonaws.com mybucket KMXXEQJBMJVK965E z7u4pxRSwCHji+wl1sNqDAeDVvc3AHo6RqVbC7Ur5XvexPnCyhg0bfeNbqzDUjepYdYKt3iuGwE= " Rgds. -- MSc Heitor Faria (Miami/USA) CEO Bacula LatAm mobile1: + 1 909 655-8971 mobile2: + 55 61 98268-4220 América Latina [ http://bacula.lat/]___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-sd can't upload to AWS S3
Hi Radosław Korzeniewski, Yes, SELinux is enabled. I've disabled it but the result is the same. Also, I've stopped the local firewall and got the same result. Tks Otaci Martins otaci Em ter., 21 de set. de 2021 às 04:43, Radosław Korzeniewski < rados...@korzeniewski.net> escreveu: > Hello, > > pon., 20 wrz 2021 o 21:29 Otaci Martins > napisał(a): > >> bacula-sd can't upload volumes to AWS S3 if running as systemd daemon, >> but if called directly by terminal cli the upload works perfectly. >> >> 1. Steps to reproduce the upload error: >> --- >> $ sudo systemctl start bacula-sd >> >> $ ps -ef | grep bacula >> root 12573 1 0 15:39 ?00:00:00 /opt/bacula/bin/bacula-sd >> -fP -c /opt/bacula/etc/bacula-sd.conf >> root 13555 1 0 Sep15 ?00:00:01 /opt/bacula/bin/bacula-fd >> -fP -c /opt/bacula/etc/bacula-fd.conf >> bacula 29641 1 0 10:58 ?00:00:00 >> /opt/bacula/bin/bacula-dir -fP -c /opt/bacula/etc/bacula-dir.conf >> >> $ echo "cloud list storage=S3CLOUD" | bconsole >> Connecting to Director 192.168.10.12:9101 >> 1000 OK: 10002 BACULA11-DIR Version: 11.0.5 (03 June 2021) >> Enter a period to cancel a command. >> cloud list storage=S3CLOUD >> Automatically selected Catalog: BACULA11-CA >> Using Catalog "BACULA11-CA" >> Connecting to Storage daemon S3CLOUD at 192.168.10.12:9103 ... >> 3900 Error cannot get cloud Volume list. ERR=S3_list_bucket >> ERR=FailedToConnect CURL Effective URL: >> https://mybucket.s3.sa-east-1.amazonaws.com/?delimiter=%2F CURL OS >> Error: 110 CURL Effective URL: >> https://mybucket.s3.sa-east-1.amazonaws.com/?delimiter=%2F CURL OS >> Error: 110 >> > > The OS Error 110 (AFAIK) is a "Connection Timeout". Do you have SELinux > enabled? If yes then test with selinux disabled. > > best regards > -- > Radosław Korzeniewski > rados...@korzeniewski.net > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-sd can't upload to AWS S3
Hello, pon., 20 wrz 2021 o 21:29 Otaci Martins napisał(a): > bacula-sd can't upload volumes to AWS S3 if running as systemd daemon, but > if called directly by terminal cli the upload works perfectly. > > 1. Steps to reproduce the upload error: > --- > $ sudo systemctl start bacula-sd > > $ ps -ef | grep bacula > root 12573 1 0 15:39 ?00:00:00 /opt/bacula/bin/bacula-sd > -fP -c /opt/bacula/etc/bacula-sd.conf > root 13555 1 0 Sep15 ?00:00:01 /opt/bacula/bin/bacula-fd > -fP -c /opt/bacula/etc/bacula-fd.conf > bacula 29641 1 0 10:58 ?00:00:00 /opt/bacula/bin/bacula-dir > -fP -c /opt/bacula/etc/bacula-dir.conf > > $ echo "cloud list storage=S3CLOUD" | bconsole > Connecting to Director 192.168.10.12:9101 > 1000 OK: 10002 BACULA11-DIR Version: 11.0.5 (03 June 2021) > Enter a period to cancel a command. > cloud list storage=S3CLOUD > Automatically selected Catalog: BACULA11-CA > Using Catalog "BACULA11-CA" > Connecting to Storage daemon S3CLOUD at 192.168.10.12:9103 ... > 3900 Error cannot get cloud Volume list. ERR=S3_list_bucket > ERR=FailedToConnect CURL Effective URL: > https://mybucket.s3.sa-east-1.amazonaws.com/?delimiter=%2F CURL OS Error: > 110 CURL Effective URL: > https://mybucket.s3.sa-east-1.amazonaws.com/?delimiter=%2F CURL OS Error: > 110 > The OS Error 110 (AFAIK) is a "Connection Timeout". Do you have SELinux enabled? If yes then test with selinux disabled. best regards -- Radosław Korzeniewski rados...@korzeniewski.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] bacula-sd can't upload to AWS S3
bacula-sd can't upload volumes to AWS S3 if running as systemd daemon, but
if called directly by terminal cli the upload works perfectly.
1. Steps to reproduce the upload error:
---
$ sudo systemctl start bacula-sd
$ ps -ef | grep bacula
root 12573 1 0 15:39 ?00:00:00 /opt/bacula/bin/bacula-sd
-fP -c /opt/bacula/etc/bacula-sd.conf
root 13555 1 0 Sep15 ?00:00:01 /opt/bacula/bin/bacula-fd
-fP -c /opt/bacula/etc/bacula-fd.conf
bacula 29641 1 0 10:58 ?00:00:00 /opt/bacula/bin/bacula-dir
-fP -c /opt/bacula/etc/bacula-dir.conf
$ echo "cloud list storage=S3CLOUD" | bconsole
Connecting to Director 192.168.10.12:9101
1000 OK: 10002 BACULA11-DIR Version: 11.0.5 (03 June 2021)
Enter a period to cancel a command.
cloud list storage=S3CLOUD
Automatically selected Catalog: BACULA11-CA
Using Catalog "BACULA11-CA"
Connecting to Storage daemon S3CLOUD at 192.168.10.12:9103 ...
3900 Error cannot get cloud Volume list. ERR=S3_list_bucket
ERR=FailedToConnect CURL Effective URL:
https://mybucket.s3.sa-east-1.amazonaws.com/?delimiter=%2F CURL OS Error:
110 CURL Effective URL:
https://mybucket.s3.sa-east-1.amazonaws.com/?delimiter=%2F CURL OS Error:
110
2. Steps to reproduce the upload success:
-
$ sudo systemctl stop bacula-sd
$ sudo /opt/bacula/bin/bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf
$ ps -ef|grep bacula
root 13362 11959 0 15:53 pts/100:00:00 sudo
/opt/bacula/bin/bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf
root 13364 13362 0 15:53 pts/100:00:00 /opt/bacula/bin/bacula-sd
-fP -c /opt/bacula/etc/bacula-sd.conf
root 13555 1 0 Sep15 ?00:00:01 /opt/bacula/bin/bacula-fd
-fP -c /opt/bacula/etc/bacula-fd.conf
bacula 29641 1 0 10:58 ?00:00:00 /opt/bacula/bin/bacula-dir
-fP -c /opt/bacula/etc/bacula-dir.conf
$ echo "cloud list storage=S3CLOUD" | bconsole
Connecting to Director 192.168.10.12:9101
1000 OK: 10002 BACULA11-DIR Version: 11.0.5 (03 June 2021)
Enter a period to cancel a command.
cloud list storage=S3CLOUD
Automatically selected Catalog: BACULA11-CA
Using Catalog "BACULA11-CA"
Connecting to Storage daemon S3CLOUD at 192.168.10.12:9103 ...
++---+--+--+---+
|Volume Name | Status | Media Type | Pool
|VolBytes |
++---+--+--+---+
| AWSS3-0076 | Full | CloudAWS |
AWSS3-PO | 8.528 MB |
|BACKUP-CATALOGO |Append | CloudAWS |
AWSS3-PO | 1.015 MB |
++---+--+--+---+
3. My question:
---
So, once the config files are exactly the same in the two scenarios, what
can i search for?
4. My specs:
Bacula CE 11.0.5 (installed from official rpm repos)
SO:
CentOS Linux release 7.9.2009 (Core)
Linux 3.10.0-1160.15.2.el7.x86_64 #1 SMP Wed Feb 3 15:06:38 UTC 2021 x86_64
x86_64 x86_64 GNU/Linux
bacula-dir.conf:
Storage {
Name = S3CLOUD
Address = 192.168.10.12
SDPort = 9103
Password = "bacula"
Device = AWSS3-DRIVE-01
Media Type = CloudAWS
Maximum Concurrent Jobs = 10
}
bacula-sd.conf:
Device {
Name = AWSS3-DRIVE-01
Device Type = Cloud
Cloud = AWSS3
Archive Device = /tmp/hdd03
Maximum Part Size = 1 GB
Maximum File Size = 1 GB
Media Type = CloudAWS
LabelMedia = yes
Random Access = yes
AutomaticMount = yes
RemovableMedia = no
AlwaysOpen = no
}
Cloud {
Name = AWSS3
Driver = "S3"
HostName = "s3.sa-east-1.amazonaws.com"
BucketName = "MyBucket"
AccessKey = "MyAccessKey"
SecretKey = "MySecretKey"
Protocol = HTTPS
UriStyle = VirtualHost
Truncate Cache = No
Upload = EachPart
Region = "sa-east-1"
MaximumUploadBandwidth = 5MB/s
}
Systemd bacula-sd.service:
[Unit]
Description=Bacula Storage Daemon service
Requires=network.target
After=network.target
RequiresMountsFor=/opt/bacula/working /opt/bacula/etc /opt/bacula/bin
[Service]
Type=simple
User=root
Group=disk
ExecStart=/opt/bacula/bin/bacula-sd -fP -c /opt/bacula/etc/bacula-sd.conf
SuccessExitStatus=15
LimitMEMLOCK=infinity
[Install]
WantedBy=multi-user.target
---
Thanks,
Otaci Martins
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
