Re: [Bacula-users] cloud_dev could not open Cloud driver type=1 for Device=CloudStorage
> On Mon, 1 Jul 2019 13:11:07 +0200, Kern Sibbald said: > > Do you see any other alternative for a C/C++ than to use libs3? Have you already ruled out using the official SDK (https://aws.amazon.com/sdk-for-cpp/)? __Martin ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] 9.4.3/9.4.4 problem with full backups
On 6/3/19 4:38 PM, Phil Stracchino wrote: > On 6/3/19 10:40 AM, Phil Stracchino wrote: >> Any suggestions? Short of rolling back to 9.4.2? > > So, I rolled back to 9.4.2. Smooth tape change, no catatonic SD, no > Director crash, and 9.4.2 wrote 812GB to the tape that 9.4.4 declared > full at 352GB. Monthly backup day again. No lock-up, no prematurely-full tapes so far, and I'd swear I'm seeing better throughput. I don't know how to narrow down precisely where it is, but I'm increasingly convinced there was some kind of Director regression in 9.4.3/9.4.4. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] cloud_dev could not open Cloud driver type=1 for Device=CloudStorage
On 01.07.19 13:11, Kern Sibbald wrote: > Yes, this is a problem. I remember we discussed libs3 some time ago, > but I did not realize that it has a number of security issues. I am no C/C++ wizard but just the fact that I need to disable about all checks and warnings a modern compiler has to even get the code to compile tells me as much as I need to know. Also looking at the commits after December 2016 tells the same, as there are several commits fixing memory leaks and buffer overflows. > Do you see any other alternative for a C/C++ than to use libs3? I'm sorry, unfortunately everybody seems to have forked and embedded libs3 at some point in the past and just uses that version for their projects, with all the problems it contains. Everyone else seems to use Rust or Go for their S3-based projects. :) > For other S3 vendors, Bacula Systems has resorted in using the vendor > supplied command line tools to access the their clouds. This gets > around the incompatibility of S3 implementations problems, but it opens > new problems in that the vendors typically supply binaries, and if they > have problems or bugs there is no way to fix them. So far, I have no > good solution to the problem. I would be interested in any of your > suggestions. I am afraid I really have none at the moment. > In fact, if I thought there were enough Bacula users using AWS S3, I > would even consider fixing and maintaining the libs3 package myself. Of > course that would be workable only if Debian and perhaps other vendors > would adopt such a project. The Debian policy allows for embedding libraries in special circumstances, for example if the forked code diverges enough from the library already in the Debian repository that using the existing one is difficult or impossible. (Besides, the current libs3 has no reverse dependencies and is essentially useless and should be removed.) There should be no problems if you fork libs3, fix it to meet modern security and coding standards and just embed it into Bacula to be statically linked to the SD or just the plugin. But since I am not DD/DM, take my words with caution. Grüße, Sven. signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] cloud_dev could not open Cloud driver type=1 for Device=CloudStorage
Hello Sven, Yes, this is a problem. I remember we discussed libs3 some time ago, but I did not realize that it has a number of security issues. I do know that it is no longer maintained by the original maintainer. Bacula Systems has been updating the source code as changes are needed to make our cloud driver work. Do you see any other alternative for a C/C++ than to use libs3? For other S3 vendors, Bacula Systems has resorted in using the vendor supplied command line tools to access the their clouds. This gets around the incompatibility of S3 implementations problems, but it opens new problems in that the vendors typically supply binaries, and if they have problems or bugs there is no way to fix them. So far, I have no good solution to the problem. I would be interested in any of your suggestions. In fact, if I thought there were enough Bacula users using AWS S3, I would even consider fixing and maintaining the libs3 package myself. Of course that would be workable only if Debian and perhaps other vendors would adopt such a project. Best regards, Kern On 6/30/19 5:17 PM, Sven Hartge wrote: On 30.06.19 16:57, Sven Hartge wrote: We would need to ship the special libs3 either as embedded code inside the Bacula source package or package it as a separate libs3-bacula package. Both will get an instant veto from the Security Team and the Release Managers. (Debian Policy §4.13) The main problem here is that libs3 is already in Debian, but in the ancient version 2.0 which doesn't even work with today's S3 anymore. (It really should be removed from Debian, to be honest.) After version 2.0, there have no new official releases been made by the upstream developer and looking at the commit history on Github it seems largely abandoned, aside from some merges PRs here and there. When Bacula Systems implemented the Cloud driver in 2016.12.09 it used the then git-HEAD of the library but it was still in flux back then, so any version a bit older or a bit newer will not work with Bacula, so even if Debian shipped a newer version, it would be incompatible with Bacula. Now, that special version from 2016.12.09 still contains some very jarring problems including buffer overflows or memory leaks. No distribution would want to support that code. But we already went over this back when 9.4.0 was released and Carsten and I tried to package the Cloud driver for Debian but gave up when it dawned on us what an ordeal it would be to achieve this in the constraints of Debian. Grüße, Sve. ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
